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Abstract 

In  order  to  understand  why  organizations  make  certain  decisions  and  target 
certain  outcomes,  it  is  useful  to  understand  their  priorities  and  preferences,  commonly 
referred  to  as  “values.”  This  research  explores  the  relationship  between  the  technical 
values  held  by  system  development  teams  and  the  operational  effectiveness  of  the 
systems  those  teams  produce.  Specifically,  it  examines  the  impact  of  a  value  set  called 
FIST  (Fast,  Inexpensive,  Simple,  Tiny)  on  DoD  and  NASA  system  development  projects, 
and  investigates  the  correlation  between  the  FIST  values  and  operational  outcomes. 

The  findings  show  that  the  FIST  value  set  enhances  project  stability,  increases  the 
project  leader’s  control  and  accountability,  optimizes  failure,  fosters  “luck,”  and 
facilitates  learning.  These  benefits  of  the  FIST  approach  all  support  the  goal  of  ensuring 
the  organization  delivers  systems  which  are  “available  when  needed  and  effective  when 
used.”  FIST  is  therefore  recommended  as  an  effective  approach  to  system  development, 
and  several  heuristics  are  provided  to  facilitate  understanding  and  application  of  these 
values. 
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THE  EFFECT  OF  VALUES 


ON  TECHNOLOGY  DEVEOPMENT  PROJECT  OUTCOMES 


Every  gun  that  is  made,  every  warship  launched,  every  rocket  fired  signifies,  in  the  final 
sense,  a  theft  from  those  who  hunger  and  are  not  fed,  those  who  are  cold  and  not  clothed. 
We  pay  for  a  single  fighter  plane  with  a  half-million  bushels  of  wheat.  We  pay  for  a 
single  destroyer  with  new  homes  that  could  have  housed  more  than  8,000  people. 

-  President  Dwight  D.  Eisenhower,  April  19,  1953 


I.  Introduction 

This  research  project  explores  the  relationship  between  the  organizational  values 
held  by  system  development  teams  and  the  operational  outcomes  of  the  systems  those 
teams  and  organizations  produce.  Specifically,  it  examines  the  impact  of  a  set  of  values 
called  FIST,  which  stands  for  Fast,  Inexpensive,  Simple  and  Tiny,  asking  to  what  extent 
(and  under  what  conditions)  the  FIST  values  contribute  to  the  development  and  delivery 
of  effective,  reliable  systems.  It  then  offers  some  suggestions  on  ways  to  implement  this 
value  set,  in  order  to  improve  the  operational  perfonnance  of  military  development 
projects. 

FIST  Origins 

In  2002,  when  I  was  a  junior  Captain  assigned  to  the  National  Imagery  and 
Mapping  Agency  (now  called  the  National  Geospatial-Intelligence  Agency),  I  began  an 
informal,  independent  investigation  into  high-perfonnance  military  technology 
development  programs.  Intrigued  by  organizational  behavior  research  and  the  work  of 
management  gurus  like  Peter  Drucker  and  Tom  Peters,  I  examined  the  groups  and 
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organizations  responsible  for  developing  successful  technologies  and  weapons.  I  also 
investigated  the  factors  leading  to  failed  projects. 

The  early  research  was  informal  but  rigorous.  It  focused  on  analyzing  spectacular 
failures  like  the  Navy’s  A-12  Avenger  and  prominent  successes  like  the  Air  Force’s  F-16 
Falcon.  The  cases  I  selected  turned  out  to  be  what  Flyvbjerg  refers  to  as  critical  cases, 
extreme  cases  and  paradigmatic  cases  (Flyvbjerg,  2006:230),  although  I  would  not 
encounter  those  tenns  until  some  years  later.  Flyvbjerg  argues  for  the  importance  of  using 
atypical  cases  in  building  and  testing  theories,  explaining  “Atypical  or  extreme  cases 
often  reveal  more  infonnation  because  they  activate  more  actors  and  more  basic 
mechanisms  in  the  situation  studied.”  (Flyvbjerg,  2006:229) 

As  the  research  progressed,  I  quickly  discovered  that  many  researchers  and 
writers  had  explored  various  pieces  of  the  defense  acquisition  puzzle  and  produced  a 
bewilderingly  large  set  of  recommended  solutions  which  were  often  overlapping, 
mutually  exclusive  and/or  internally  contradictory.  Further,  many  of  the  reform  efforts 
seemed  to  focus  solely  on  improving  programmatic  outcomes  (cost  and  schedule 
performance)  rather  than  operational  effectiveness.  I  was  not  satisfied  that  anyone  had 
provided  a  comprehensive-but-feasible  prescription  for  actually  improving  the 
operational  effectiveness  of  defense  acquisition  projects.  I  also  did  not  find  much 
evidence  that  the  recommendations  led  to  the  desired  results,  either  programmatically  or 
operationally. 

Some  sources,  such  as  the  1986  Packard  Commission  or  Ross  McNutt’s  1989 
PhD  thesis  at  MIT,  focused  on  excessive  timelines  as  the  central  problem  in  military 
technology  development  efforts.  A  few  (Fitzsimonds  and  van  Tol,  1994)  explored  the 
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relationship  between  budget  size  and  operational  innovation,  concluding  that  “innovation 
is  not  necessarily  or  even  primarily  a  function  of  budget.  Many...  innovations  came  at  a 
time  of  low  budgets...”  (Fitzsimonds  and  van  tol,  1994:29).  Others  discussed  the  role  of 
complexity,  both  organizational  and  technological.  Over  a  period  of  several  years,  the 
work  of  these  and  other  researchers,  along  with  my  own  independent  observations,  began 
to  coalesce  into  something  resembling  a  unified  theory  of  effective  military  system 
development,  which  focused  on  developing  systems  that  would  be  available  when  needed 
and  effective  when  used. 

In  a  series  of  articles  published  by  Defense  Acquisition  University,  my  colleague 
Maj.  Chris  Quaid  and  I  outlined  the  four  elements  of  what  has  become  the  FIST  model 
for  system  development.  The  series  culminated  in  a  capstone  article  titled  FIST,  Part  5, 
which  was  published  in  Defense  AT&L  ’s  May/June  2006  issue.  This  article  summarized 
and  synthesized  our  previous  research  findings,  and  offered  the  FIST  value  set  as  an 
effective  way  to  deliver  innovative,  reliable  technology  systems  to  military  customers, 
with  a  firm  focus  on  operational  effectiveness  as  the  primary  measure  of  merit. 

Much  of  our  original  FIST  research  up  to  that  point  was  based  on  personal 
experience  and  observation,  and  was  admittedly  anecdotal.  However,  the  FIST  model 
was  also  built  upon  rigorous,  academically-validated  research  performed  and  published 
by  others,  including  several  of  the  previously  mentioned  sources,  along  with  reports  by 
the  Government  Accountability  Office,  other  researchers  across  the  services  and 
academia,  and  several  congressional  blue  ribbon  panels. 

Upon  hearing  of  our  approach,  Dr.  Alexander  Laufer  of  Israel’s  Technion 
University  (and  the  former  Editor-in-Chief  of  NASA’s  program  management  journal 
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ASK)  described  Maj  Quaid  and  I  as  “reflective  practitioners,”  a  label  we  immediately 
embraced. 

The  tenn  reflective  practitioner  comes  from  Donald  Schon’s  book  The  Reflective 
Practitioner,  published  in  1984.  Reflective  practice  generally  refers  to  a  research 
methodology  involving  the  deliberate  examination  of  one’s  experiences  combined  with 
formal  academic  knowledge  in  order  to  establish  deeper  understanding  of  the  practice  in 
question.  Today,  this  methodology  is  most  often  used  in  the  fields  of  medicine  and 
education,  but  Schon’s  book  explicitly  applies  it  to  practitioners  from  a  wide  range  of 
fields,  including  architecture,  engineering  and  management.  In  this  thesis  paper,  the 
practitioners  of  interest  are  referred  to  as  “project  leaders,”  a  general  term  that  includes 
program  managers,  systems  engineers,  system  architects,  system  operators,  commanders 
and  other  people  responsible  for  leading  the  development  of  a  technology  system. 

Schon  argues  that  “competent  practitioners  usually  know  more  than  they  can  say. 
They  exhibit  a  kind  of  knowing-in-practice,  most  of  which  is  tacit.”  (Schon,  1 984: viii). 
This  is  counter  to  the  prevailing  Positivist  academic  worldview,  which  tends  to  operate 
under  “the  conviction  that  empirical  science  was  not  just  a  fonn  of  knowledge,  but  the 
only  source  of  positive  knowledge  in  the  world.”  (Schon,  1984:32).  The  initial  FIST 
research  clearly  belongs  in  the  realm  of  reflective  practice,  since  its  foundation  largely 
rests  on  “knowing-in-practice,”  and  was  bolstered  by,  rather  than  established  on, 
academically  oriented  studies. 

Schon  goes  on  to  show  that  practitioners  often  have  “a  capacity  for  reflection  on 
their  intuitive  knowing  in  the  midst  of  action. . .  [which  is]  susceptible  to  a  kind  of  rigor 
that  is  both  like  and  unlike  the  rigor  of  scholarly  research.”  (Schon,  1984:ix)  This 
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intellectual  rigor,  which  is  “both  like  and  unlike”  academic  rigor,  is  precisely  what  we 
aimed  to  achieve  in  our  initial  research.  Our  experience  resonates  strongly  with  Schon’s 
description  that  “our  knowledge  is  in  our  action.”  (Schon,  1984:49,  emphasis  in  original). 

Thus,  the  FIST  model  has  its  roots  in  years  of  reflective  practice,  although  the 
formal  term,  like  Flyvberg’s  case  categories,  was  something  we  encountered  only  after 
several  years  of  actually  doing  it.  This  thesis  is  therefore  the  continuation  of  a  multi-year 
research  project  that  began  as  the  result  of  reflective  practice.  It  aims  to  produce  an 
original  academic  assessment  of  the  FIST  model’s  applicability  and  limitations,  and  to 
inject  a  more  fonnal  type  of  academic  rigor  into  our  previous  reflective  analyses. 

What  are  Values? 

Values  are  statements  of  priorities  and  preferences.  An  organization’s  values 
typically  answer  the  questions  “What  is  important?”  or  “What  is  good?”  The  answers  to 
these  questions  shape  the  measures  of  merit  project  leaders  use  to  assess  and  shape  a 
system  development  project.  When  an  organization  expresses  the  FIST  values,  they  are 
stating  “It  is  good  and  important  to  be  fast,  inexpensive,  simple  and  tiny.”  The  technical 
measures  of  merit  driven  by  the  FIST  values,  therefore,  are  oriented  around  low-cost, 
short  schedules,  simplicity  and  smallness. 

Values  are  primarily  cultural  artifacts,  and  their  existence  within  an  organization 
is  determined  by  the  presence  of  value  clues.  Value  clues  may  be  measurable,  taking  the 
form  of  physical,  procedural,  fonnal  or  otherwise  visible  features,  but  are  also  often 
intangible,  taking  the  form  of  an  unmeasurable  “sense”  that  the  particular  value  is 
important  and  good. 
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Thus,  values  can  be  either  explicit  or  implicit.  Just  as  there  is  a  difference  between 
an  organization’s  intended  strategy  and  the  realized  strategy,  there  is  also  often  a 
difference  between  stated  values  (which  are  often  idealized)  and  the  actual  values  which 
are  put  into  practice  by  the  organization’s  members. 

Organizational  behaviorists  point  out  that  an  organization’s  realized  strategy  is  a 
combination  of  stated  and  emergent  strategies.  Similarly,  the  realized  values  are  a 
combination  of  intended  values  (which  the  organization  aims  to  adopt)  and  emergent 
values  (which  the  organization  actually  demonstrates  by  its  actions  and  behavior).  This  is 
important  because  while  an  individual  within  an  organization  may  make  statements 
supporting  a  particular  value,  these  statements  will  need  to  be  measured  against  actual 
behavior  and  decisions.  That  is,  the  value  clues  hidden  in  the  project  leader’s  actions  need 
to  be  considered  when  assessing  an  organization’s  values,  alongside  whatever  explicit 
value  statements  the  individual  or  other  organizational  leaders  might  make.  Implicit  or 
unconscious  values  may  have  a  significant  effect  on  the  organization’s  decision-making 
process  and  on  the  project’s  eventual  perfonnance. 

Why  Look  at  Values? 

Values  guide  and  shape  organizational  behavior  across  the  entire  spectrum  of 
decision  making,  from  requirements  definition  and  system  design  to  organizational 
structure  and  process  establishment.  In  fact,  “Values  are  often  considered  the  bedrock  of 
corporate  culture.  Many  corporate  leaders  have  validated  these  observations  about  the 
central  importance  of  organizational  values.”  (Posner,  Kouzes,  Schmidt,  1985:  293).  So, 
if  we  want  to  understand  why  an  organization  makes  certain  decisions  (which  lead  to 
particular  outcomes),  it  is  important  to  understand  the  organization’s  values. 
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A  paper  titled  Strategic  Planning  101  points  out  values  underlie  the  decision¬ 
making  process.  Ignoring  the  values  of  an  organization  is  disastrous  for  a  strategic  plan 
because  regardless  of  the  plan,  major  and  minor  decisions  will  align  with  the  culture  but 
not  necessarily  with  the  mission.”  (Workstar,  2008). 

According  to  Seevers,  “Values  are  enduring  because  they  are  neither  completely 
stable  or  unstable,  but  rather  change  in  accordance  to  the  changing  physical,  social,  and 
spiritual  environments  of  the  individuals  and  groups  that  embrace  them.”  (Seevers, 
2000:71).  This  blending  of  stability  and  openness  to  change  gives  values  a  dynamic, 
powerful  ability  to  resist  irrelevance  or  obsolescence,  making  them  an  important 
framework  for  understanding  an  organization’s  behavior. 

Values  have  an  enormous  impact  on  system  design,  from  requirements  definition 
to  the  final  operational  testing,  and  weapon  system  design  is  no  exception.  As  Tobias  et 
al  point  out,  “Weapons,  too,  reflect  the  societies  that  produce  them.”  (Tobias  et  al, 
1982:382).  The  authors  briefly  compare  Russian  and  German  tanks  from  World  War  II: 

One  of  the  best  Gennan  tanks  of  World  War  II  was  the  Panzer  IV.  It  represented 
not  only  the  high  state  of  German  technology,  engineering  skill,  and  precision 
workmanship,  but  the  Blitzkrieg  style  of  war  as  well.  The  best  Russian  tank  of  the 
period,  the  T-34,  was  equally  typical  of  Soviet  values.  The  T-34  was  simply 
designed  and  easily  manufactured,  but  eminently  reliable  (particularly  well- 
adapted  to  the  Russian  winter)  and  cheap.  Moreover,  being  easy  to  operate  and 
maintain,  it  could  be  manned  by  technically  unsophisticated  crews. . .  Despite  their 
simplicity,  the  T-34’s  generally  prevailed  over  the  Panzers.”  (Tobias  et  al, 
1982:382). 

Thus,  we  see  that  a  country’s  values  tend  to  influence  both  its  style  of  warfighting 
and  the  shape  of  its  weaponry.  American  values  are  examined  in  the  Discussion  section 
of  this  paper. 
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The  FIST  Value  Set 

The  FIST  value  set  was  summarized  in  a  capstone  article  by  Quaid  and  Ward 

titled  FIST:  Part  5  (Quaid  &  Ward,  2006a).  They  explain: 

The  components  of  FIST  are,  first  and  foremost,  statements  of  professional 
values.  They  are  characteristics,  attributes,  or  entities  that  are  judged  to  be  of 
greater  worth  than  the  alternatives.  They  describe  principles,  standards,  and 
qualities  that  are  deemed  worthwhile  and  desirable...  Like  any  set  of  values,  FIST 
can  be  understood  as  a  collection  of  philosophical  assertions,  designed  to  drive 
actions  and  infonn  decision  making.  (Quaid  &  Ward,  2006a:30-31). 

Although  the  FIST  value  set  has  four  parts,  it  is  properly  understood  as  a  single 

idea.  An  attempt  to  remove  some  portion  of  this  value  set  is  likely  to  impact  the  program 

manager’s  ability  to  implement  any  of  it  at  all.  This  attribute  of  singularity  is  similar  to 

Laufer’s  explanation  of  a  set  of  principles  in  his  book  Breaking  The  Code  of  Program 

Management  (Laufer,  2009).  Laufer  writes: 

The  implementation  of  any  one  principle  and  its  impact  on  project  success 
depends  on  the  implementation  of  all  the  others.  In  order  to  compensate  for  the 
inability  to  fully  adhere  to  a  principle,  be  prepared  to  modify  the  implementation 
of  the  others  as  well  as  to  adjust  project  expectations.  (Laufer  2009: 183). 

Accordingly,  this  research  treats  the  FIST  value  set  as  a  single  entity,  in  which  the 

neglect  of  any  one  component  will  impede  an  organization’s  ability  to  implement  the 

remaining  components.  For  example,  decreasing  a  program’s  budget  while 

simultaneously  increasing  the  schedule  is  inherently  untenable,  because  it  takes  time  to 

spend  money  and  it  takes  money  to  spend  time.  Similarly,  emphasizing  speed  while 

increasing  the  system’s  complexity  leads  to  contradiction  and  reduces  the  ability  of  the 

team  to  fully  express  either  value. 

Properly  understood,  the  FIST  values  are  mutually  reinforcing,  not  mutually 
exclusive.  Organizations  which  attempt  to  express  the  FIST  values  often  discover  that 
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implementing  one  of  the  values  tends  to  improve  the  organization’s  ability  to  implement 
the  other  values.  As  Spinney  explains,  “Operating  costs  are  strongly  related  to  the  size, 
technical  complexity  and  operating  tempo  of  the  military  forces.  Increasing  the 
complexity  of  weapons  increases  the  size  and  complexity  of  the  support  tail,  and 
operating  costs  go  up...”  (Spinney,  1985:185).  The  inverse  of  Spinney’s  observation  is 
also  true  -  reductions  in  a  system’s  complexity  tends  to  decrease  the  size  and  complexity 
of  the  support  tail,  so  operating  costs  go  down,  and  time  to  delivery  is  decreased. 

The  programs  evaluated  in  this  research  are  given  a  score  in  each  of  the  four 
categories,  an  F-score,  I-score,  S-score  and  a  T-score.  The  four  scores  are  compiled  into 
an  overall  FIST  score,  which  is  then  compared  with  the  system’s  operational  outcome. 
The  following  section  introduces  each  of  the  four  FIST  components. 

F:  FASTIS  GOOD 

Since  the  mid- 1 980 ’s,  speed  has  become  essential  to  business — to  such  an  extent 
that  many  authorities  argue  that  speed  is  the  single  most  significant  basis  for 
competitive  advantage  in  the  years  ahead.  (Laufer  2009:33) 

The  F  in  FIST  stands  for  Fast.  Organizations  often  express  this  value  by  taking 

steps  to  encourage  and  reward  short  development  timelines.  These  steps  can  include 

strong,  explicit,  frequent  affirmations  of  the  importance  of  speed  from  senior  leaders, 

formal  contractual  commitments  to  maintaining  deadlines  and  concrete  steps  taken  to 

actively  reduce  planned  development  timelines.  In  an  environment  where  Fast  is  valued, 

participants  are  rewarded  for  meeting  or  beating  contract  deadlines,  milestones  and 

delivery  dates.  When  Fast  is  not  valued,  schedules  tend  to  be  very  long,  and  requests  for 

schedule  slips  are  viewed  as  a  reasonable  approach  to  correcting  difficulties. 
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This  aspect  was  introduced  in  an  article  titled  “It’s  About  Time”  (Quaid  and  Ward, 

2006b).  The  article  begins  with  a  look  at  some  historical  studies  and  pronouncements, 

and  points  out  that  for  over  twenty  years,  there  has  been  a  wide  agreement  that,  in  the 

words  of  the  1986  Packard  Commission,  “An  unreasonably  long  acquisition  cycle  ...  is  a 

central  problem  from  which  most  other  acquisition  problems  stem.”  The  article  provides 

several  other  statements  in  support  of  the  importance  of  speed: 

1986:  “Many  have  come  to  accept  the  ten-to-fifteen  year  acquisition  cycle  as 
normal  ...  We  believe  that  it  is  possible  to  cut  this  cycle  time  in  half.'"  — Packard 
Commission  Report 

1986:  “The  most  important  way  technology  could  enhance  our  military  capability 
would  be  to  cut  the  acquisition  cvcle  time  in  half."  — Chairman  of  the  Joint  Chiefs 
of  Staff 

1994:  “Deliver  emerging  technology  to  troops  in  50%  less  time."  — Federal 
Acquisition  Streamlining  Act  (FASA) 

2003:  “advanced  technology  shall  be  integrated  into  producible  systems  and 
deployed  in  the  shortest  time  practicable.”  -May  2003  update  to  DoDD  5000. 1 

Dr.  Alexander  Laufer  similarly  examined  a  series  of  studies  on  the  issue  of  project 

timelines,  and  came  to  a  similar  conclusion  about  the  impact  of  excessive  timelines  and 

schedule  delays.  He  writes  “while  an  engineering  cost  overrun  of  50  percent  impacts 

overall  profitability  by  just  4  percent,  a  time  delay  of  six  months  in  project  completion 

can  result  in  a  32  percent  loss  in  after-tax  profit.”  (Laufer,  2009:33).  The  data  provides 

clear  support  for  the  Fast  value,  and  yet  this  value  is  often  absent  from  project  teams. 

Along  with  helping  to  avoid  cost  impacts,  the  Fast  value  proposes  that  short 

timelines  provide: 

•  Increased  stability  in  requirements  and  funding 

•  Decreased  personnel  turnover 

•  Greater  accountability 
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•  More  accurate  user  feedback  (build  the  system  quickly,  field  it  quickly,  gain 
feedback  quickly,  repeat. . .) 

•  Smaller  budgets  (because  time  =  money) 

•  Closer  match  to  the  operational  environment’s  needs 

•  Decreased  risk  of  obsolescence 

•  More  meaningful  metrics  (less  time  for  collection  and  analysis  of  pointless 
metrics,  decreased  risk  of  collecting  and  analyzing  out-of-date  metrics) 

•  Enhanced  opportunities  for  learning  from  experience 

These  are  some  of  the  more  prominent  reasons  given  for  identifying  speed  as 
“important  and  good.”  Program  managers  who  hold  to  the  FIST  values  therefore 
explicitly  embrace  the  value  of  speed,  and  take  concrete  steps  to  demonstrate  the  belief 
that  being  fast  matters.  Projects  where  speed  is  clearly  identified  as  “important  and  good” 
have  a  high  F-score. 

A  question  which  often  comes  to  mind  in  discussions  about  the  Fast  value 
revolves  around  the  difficulty  of  assessing  the  time  duration  of  a  system  development 
project.  When  should  we  start  and  stop  the  clock?  Did  the  project  begin  when  the  Request 
For  Proposals  was  sent  out,  when  the  contract  was  signed,  or  at  some  other,  perhaps 
earlier,  point?  How  far  back  into  the  basic  research  process  should  one  go?  And  on  the 
other  end  of  the  timeline,  should  we  stop  the  clock  when  Initial  Operational  Capability 
(IOC)  is  declared,  when  the  system  is  first  used  operationally,  or  is  there  a  better 
milestone  to  indicate  the  end  of  the  development?  It  turns  out,  these  questions  are  not 
important  in  the  context  of  this  research. 

When  assessing  the  presence  of  the  Fast  value,  it  is  not  necessary  to  have  a 
concrete,  specific  measurement  of  the  amount  of  time  spent  on  the  project.  We  must 
simply  investigate  whether  the  development  team  (government  and  contractor)  valued 
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speed,  whether  project  leaders  considered  it  “important  and  good”  to  be  fast  relative  to 
other  efforts  of  a  similar  nature.  Delivering  in  three  months  or  three  years  is  not  the  point. 
Instead,  this  research  looks  for  value  clues,  such  as  an  aggressive  schedule,  contractual 
incentives  for  early  delivery,  sacrifices  of  other  priorities  (perfonnance,  capability,  etc)  to 
ensure  timely  delivery,  or  a  willingness  to  accept  higher  levels  of  risk  rather  than  accept  a 
delay.  Ultimately,  the  Fast  value  means  that,  for  this  type  of  system  development  project, 
the  timeline  is  aggressive,  regardless  of  the  specific  amount  of  time  spent. 

Thus,  the  presence  of  the  Fast  value  is  not  determined  by  rigorously  documenting 
the  program’s  start  and  stop  times  and  comparing  it  to  some  standard.  There  is  no 
absolute  amount  of  time  which  constitutes  a  Fast  program,  although  the  GAO  notes  that  a 
development  timeline  greater  than  5  years  is  probably  too  slow  (Dodaro,  2008:26).  A 
correlation  between  values  and  programmatic  outcomes  should  not  be  assumed,  and 
adherence  to  the  Fast  value  does  not  guarantee  the  project  will  deliver  on  time.  In  fact, 
even  a  late  delivery  might  have  been  driven  by  the  Fast  value.  The  question  is  whether  or 
not  the  team  thought  it  was  important  and  good  to  be  fast,  regardless  of  whether  they 
achieved  their  schedule  goal  or  not. 

It  must  also  be  noted  that  time  pressure  and  the  Fast  value  are  not  the  same  thing. 
The  Fast  value  is  strategic  and  (perhaps  ironically)  long-term,  aimed  at  delivering  a 
functional,  sustainable  technology  solution  as  quickly  as  possible.  Mere  time  pressure,  in 
contrast,  is  often  tactical  and  short-term,  potentially  introducing  extensive  delays  down 
the  road,  in  effect  sacrificing  tomorrow  to  achieve  the  appearance  of  fast  results  today. 
This  is  reflected  in  the  saying  “take  the  time  to  do  it  right  the  first  time,  so  you  don’t  have 
to  do  it  over.” 
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The  Fast  value  does  encourage  short,  “insufficient”  timelines,  but  simply  being 
hasty  is  not  the  same  thing  as  being  Fast.  For  example,  bluntly  accelerating  an  existing 
schedule  and  cutting  corners  on  essential  tasks  is  not  consistent  with  the  Fast  value.  This 
kind  of  brute-force  approach  often  leads  to  unintended  delays  over  the  long  run,  as 
project  members  have  to  go  back  and  do  (or  re-do)  work  that  could  have  been 
accomplished  sooner  if  it  had  been  executed  correctly  in  the  earlier  attempt. 

An  article  in  Harvard  Business  Review  illustrates  this  difference  between  Fast  and 
hasty,  using  a  parable  of  a  man  pushing  an  apple  cart  to  a  distant  marketplace.  When  the 
man  asked  a  bypasser  how  long  it  would  take  to  get  to  the  market,  he  was  told  “If  you 
take  your  time,  it’ll  take  about  an  hour.  If  you  go  fast,  it’ll  take  all  day.”  (Perlow  and 
Williams,  May  2003).  This  counterintuitive  advice  makes  sense  because  the  man’s 
applecart  was  full  and  the  road  was  bumpy.  If  he  rushed,  he  would  spend  all  day  stopping 
to  pick  up  the  apples  that  bounced  out  of  his  cart.  Like  the  story  of  the  tortoise  and  the 
hare,  the  fastest  way  for  the  man  to  get  to  the  market  actually  required  him  to  walk  slowly 
(and  not  stop). 

Giving  in  to  short-tenn,  tactical  time  pressure  and  clumsily  “speeding”  tends  to 
result  in  taking  more  time,  not  less,  and  ultimately  results  in  a  slower  delivery.  Being 
hasty — i.e.  speeding — is  therefore  not  consistent  with  the  assertion  that  “it  is  important 
and  good  to  be  fast,”  because  hastiness  delays  the  ultimate  delivery  of  the  system.  The 
Fast  value,  in  contrast,  takes  a  strategic  point  of  view,  and  is  not  satisfied  with  the 
shallow  appearance  of  speed. 

As  with  many  types  of  measurement  and  assessment,  distinguishing  between 
being  Fast  and  being  hasty  can  be  difficult  to  do  in  situ,  but  is  is  possible,  particularly  for 
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project  leaders  who  are  reflective  practitioners.  If  too  many  apples  are  falling  out  of  the 
cart,  the  program  is  probably  being  hasty.  That  is,  if  the  time  allotted  is  physically 
insufficient  to  accomplish  the  activities  which  are  truly  essential,  the  program  is  speeding 
and  not  expressing  the  Fast  value.  However,  detennining  what  is  “essential”  and  figuring 
out  how  many  apples  we  can  afford  to  lose  along  the  way  is  often  a  difficult  question,  and 
is  a  matter  for  professional  judgment  and  experience. 

The  Fast  value  leads  program  managers  to  trim  unnecessary  activities.  It  also 
tempts  them  to  cut  essentials,  which  can  cause  significant  problems  and  even  cause  the 
program  to  fail.  The  question  on  the  table  is  whether  the  tool  can  be  used  effectively, 
without  injecting  undue  risk  to  the  overall  outcome.  The  best  approach  appears  to  involve 
relying  on  experienced,  talented  program  managers  who  are  wise  enough  to  distinguish 
between  necessary  and  unnecessary  activities  when  making  decisions  related  to  speed. 

Some  leaders  consider  the  risk  of  speed  to  be  acceptable.  Hector  Ruiz,  CEO  of 
AMD,  tells  his  employees  “Please,  go  get  speeding  tickets.  I  don’t  want  you  to  get 
parking  tickets.”  (Kirkpatrick,  2006:118).  This  willingness  to  metaphorically  lose  some 
apples  along  the  way,  when  done  deliberately  and  with  a  positive  view  of  failure  as  a 
learning  experience,  is  entirely  consistent  with  the  Fast  value. 

I:  INEXPENSIVE  IS  GOOD 

The  I  in  FIST  stands  for  Inexpensive.  Organizations  often  express  this  value  by 
deliberately  pursuing  low-cost  solutions  and  establishing  contractual  incentives  to  reward 
cost  under  runs.  They  often  make  fonnal  commitments  to  maintain  their  budget  and  are 
willing  to  sacrifice  other  attributes,  including  performance  or  redundancy,  to  ensure  the 
program  cost  stays  low.  When  being  Inexpensive  is  not  valued,  project  leaders  seek  to 
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increase  their  budget  as  a  means  of  solving  problems,  and  work  hard  to  ensure  that  next 
year’s  budget  is  not  decreased. 

This  piece  of  the  FIST  value  set  was  first  introduced  in  an  article  titled  “Doing 
Less  With  More,”  (Ward,  2004).  The  article  begins  with  the  assertion  that  being 
overfunded  limits  the  organization’s  ability  to  be  innovative,  and  argues  that  having  a 
small,  “insufficient”  budget  actually  leads  to  better  technology  solutions  and  superior 
operational  outcomes. 

The  Inexpensive  concept  is  based  largely  on  research  by  Pierre  Sprey  (Clark, 
1984)  and  Fitzsimonds  &  van  Tol  (Fitzsimonds  &  van  Tol,  1994).  In  a  widely  cited 
research  project,  Sprey  argued  for  the  superiority  of  “cheap  winners”  over  “expensive 
losers.”  Similarly,  Fitzsimonds  and  van  Tol’s  research  indicated  that  “militaries  are 
driven  to  innovate...  by  the  need  to  make  more  efficient  use  of  shrinking  resources...” 
They  conclude  “Innovation  is  not  necessarily  or  even  primarily  a  function  of  budget.” 
(Fitzsimonds  &  van  Tol,  1994:26,  29). 

Spinney  also  demonstrated  an  inverse  relationship  between  cost  and  effectiveness. 

In  an  analysis  of  missile  development  efforts,  he  writes: 

...  as  cost  and  complexity  increase,  fewer  missiles  are  projected  to  be  fired.  This 
implies  that  cost  is  a  major  factor  in  shaping  this  program.  Although  the  increased 
costs  are  justified  in  the  attrition  view  by  promised  increases  in  lethality,  the 
increased  costs  lead  to  less  confidence  in  predicted  lethality...  As  weapons  get 
more  expensive,  they  are  fired  less  frequently — and  less  realistically — in  training 
and  testing.  (Spinney,  1985:91). 

This  inverse  relationship  between  cost  and  effectiveness  is  one  of  the  reasons 
often  given  for  holding  to  the  Inexpensive  value.  The  components  of  the  FIST  value  set 
are  closely  interrelated  and  mutually  reinforcing.  The  behaviors  that  emerge  from  one 
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value  often  overlap  with  the  behaviors  driven  by  the  other  three  values.  Thus,  the 
Inexpensive  value  is  closely  related  with  both  Fast  and  Simple,  as  speed  and  simplicity 
are  often  pursued  for  the  sake  of  keeping  the  overall  costs  down. 

When  a  program  manager  detennines  it  is  important  and  good  to  have  a  small 
budget,  the  schedule  and  degree  of  complexity  both  naturally  tend  to  shrink,  as  does  the 
size  of  the  project  team.  Inexpensive  solutions  tend  to  leverage  existing,  mature 
technology,  rather  than  pursuing  the  more  expensive  path  of  new  development,  which  is  a 
sign  of  simplicity.  Thus,  the  use  of  mature  technology  is  a  value  clue  that  can  increase  the 
system’s  I-score  and  S-score. 

Further,  lower  costs  tend  to  require  shorter  schedules  and  reductions  in 
complexity  (both  organizationally  and  technologically).  So,  any  particular  value  clue 
statement  might  be  an  expression  of  more  than  one  component  of  FIST,  further 
supporting  the  assertion  that  FIST  is  one  idea,  not  four. 

S:  SIMPLE  IS  GOOD 

The  S  in  FIST  stands  for  Simple.  Organizations  often  express  this  value  by 
reducing  complexity  in  their  organization  and  procedures  as  well  as  in  the  technology 
systems  they  develop.  Program  managers  who  value  simplicity  tend  to  focus  their 
projects  on  a  narrow,  stable  set  of  modest  operational  requirements  and  aim  for  “the 
shrewd  application  of  available  technology,”  (AF  Studies  Board,  2008:32),  rather  than  a 
frequently  changing  and  wide-ranging  collection  of  unproven,  high-tech,  beyond-the- 
state-of-the-art  objectives.  When  Simple  is  not  valued,  project  leaders  tend  to  view 
complexity  as  a  sign  of  sophistication  and  pursue  complex,  high-technology  solutions  for 
their  own  sake,  rather  than  because  they  meet  the  actual  operational  needs. 
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This  piece  of  the  FIST  value  set  was  first  introduced  in  an  article  titled  “The 
Simplicity  Cycle,”  (Ward  2005b),  and  was  further  refined  in  a  book  of  the  same  name 
(Ward,  2005c).  Just  as  the  Fast  value  distinguishes  between  being  Fast  and  being  hasty, 
the  Simplicity  Cycle  draws  strong  distinctions  between  simplisticness  and  simplicity,  as 
well  as  between  complexity  and  complicatedness. 

The  fundamental  idea  behind  the  Simplicity  Cycle  diagram  is  that  system 
development  activities  move  through  a  cycle,  beginning  with  a  simplistic  concept  which 
becomes  more  complex  as  it  improves  and  new  capabilities  are  added.  Once  a  certain 
critical  mass  of  complexity  has  been  achieved,  optimized  solutions  are  the  result  of  a 
simplification  effort,  resulting  in  a  simple  (not  simplistic)  solution.  Any  further  increases 
in  complexity  beyond  that  critical  mass  point  actually  reduce  the  system’s  value 
(perfonnance,  reliability,  etc).  Figure  1  shows  the  Simplicity  Cycle  diagram. 


Figure  1:  The  Simplicity  Cycle 

This  diagram  illustrates  the  idea  that  optimized  simplicity  is  necessarily  built  on  a 
foundation  of  experience  and  complexity.  Using  mature,  proven  technologies  is  an 
example  of  genuine  simplicity,  building  as  it  does  on  previous  experience.  The  converse 
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is  also  true — relying  on  new  developments  and  unproven  solutions  leads  to  simplistic  or 
complicated  solutions,  which  get  a  lower  S-score.  Please  note:  simplistic  approaches 
receive  a  low  S-score,  despite  having  a  minimal  level  of  complexity,  because 
simplisticness  is  inferior  to  simplicity,  and  simplistic  approaches  do  not  indicate  the 
presence  of  the  Simple  value. 

Organizationally,  simplicity  means  relying  on  people’s  talents  and  experience  to 
achieve  objectives,  instead  of  using  elaborate,  formal  (i.e.  complex)  procedures.  A 
simplified  organization  relies  on  trust  and  talent  rather  than  erecting  intricate  mechanisms 
to  audit,  monitor,  control  and  dictate  behavior.  Simply  stated,  Theory  Y  management  is 
simpler  than  Theory  X. 

The  Simplicity  Cycle  defines  complexity  as  “consisting  of  interconnected  parts,” 
(Ward,  2007),  so  a  system  can  have  either  a  high  or  low  degree  of  complexity.  Simplicity, 
therefore,  requires  minimizing  the  number  of  components  and  interfaces  used,  whether 
those  components  be  parts  of  a  system  or  people  on  a  project  team. 

A  large  number  of  components  means  a  low  T-score,  naturally  (for  a  description 
of  the  Tiny  value,  see  the  following  section).  But  a  large  number  of  components  also 
increases  the  system’s  complexity,  and  therefore  reduces  the  project’s  S-score.  For 
example,  three  components  have  three  interfaces  and  three  possible  communication  paths. 
But  doubling  the  number  of  components  to  six  increases  the  number  of  possible 
communication  paths  to  15,  a  factor  of  5  increase,  as  shown  in  Figure  2.  Thus,  in  some 
situations,  size  has  a  more  significant  impact  on  a  system’s  S-score  than  its  T-score. 
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Figure  2:  Communication  Paths 


As  Figure  2  shows,  complex  entities — be  they  systems  or  organizations — have  a 
greater  number  of  interactions  to  manage.  This  management  has  a  cost,  naturally,  so 
complex  systems  tend  to  require  more  money  than  simple  ones  do.  McCurdy  points  out 
that  “the  ability  of  a  team  to  solve  problems  informally  is  limited  by  the  potential  number 
of  communication  channels  within  the  team.”  (McCurdy,  2001:153).  As  potential 
communication  channels  increase,  so  does  the  complexity,  while  the  ability  to  solve 
problems  is  diminished. 

McCurdy  identifies  simplicity  as  a  critical  element  in  NASA’s  Faster,  Better, 
Cheaper  initiative.  He  writes: 

Low-cost  missions  of  the  lowest  complexity...  did  very  well.  As  projects  moved 
up  the  cost/complexity  curve,  failures  occurred.  It  did  not  matter  whether  the 
more  complex  “faster,  better,  cheaper”  projects  sat  above  or  below  the  adequate — 
spending  line.  All  of  the  more  complex  projects  failed  or  were  impaired, 
regardless  of  whether  their  managers  spent  too  little  or  too  much.  (McCurdy, 
2001:27) 

McCurdy  goes  on  to  point  out  that  the  “primary  route  to  simplicity  runs  through 
size  reduction,”  (McCurdy,  2001:21),  once  again  highlighting  the  interrelated  nature  of 
the  FIST  values.  Small  is  simple  and  simple  is  small. 

Tobias  et  al  also  point  out  that  simplicity  also  tends  to  enhance  availability,  while 
complexity  degrades  it.  They  write  “complex  planes  have  more  things  that  can  go  wrong, 
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so  they  spend  more  time  in  the  shop.  As  a  result,  we  not  only  can  buy  fewer,  but  those  we 
have  are  down  for  maintenance  more  often.”  (Tobias  et  al,  1982:368).  Quoting  an 
unnamed  naval  officer,  they  further  support  the  link  between  complexity,  increased  cost 
and  diminished  availability: 

Expensive  airplanes  are  complex  airplanes,  and  complex  airplanes,  over  the  past 
ten  to  fifteen  years,  have  been  the  bane  of  our  existence.  The  costs  of  keeping  a 
stable  of  these  complex  machines  in  fighting  trim  is  astronomical.  (Tobias  et  al, 
1982:369) 

Tobias  et  al  summarize  this  into  a  simple  rule:  “as  weapons  increase  in 
complexity,  their  reliability  declines.”  (Tobias  et  al,  1982:369)  This  degradation  of 
reliability,  combined  with  increases  in  cost,  is  further  complicated  by  a  vicious  circle, 
which  Tobias  et  al  describe  this  way: 

Complexity  pushes  up  costs;  since  expenses  go  up  faster  than  available  funds, 
production  runs  decline,  which  pushes  up  unit  costs  further  still,  leaving  the 
Pentagon  with  even  fewer  weapons.  Yet,  at  the  same  time,  these  weapons  are  less 
reliable  because  of  their  complexity,  which  results  in  still  fewer  being  available 
for  service,  and  those  that  are  have  higher  maintenance  costs.”  (Tobias  et  al, 
1982:370) 

Because  integration  is  such  an  important  capability  for  virtually  every  DoD 
system,  a  project  leader’s  appreciation  for  the  systems  engineering  concept  of  high 
cohesion  /  low  coupling  can  have  a  major  impact  on  a  system’s  S-score.  High  cohesion 
and  low  coupling  can  greatly  simplify  a  system’s  development,  testing,  operations  and 
maintenance  efforts,  so  project  leaders  who  value  high  cohesion  /  low  coupling  tend  to 
get  a  higher  S-score. 

Along  with  the  principle  of  high  cohesion  /  low  coupling,  simplicity  includes  the 
concept  of  specialization  and  generalization.  That  is,  specialization  tends  to  involve 
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simplification,  as  each  component  in  the  system  is  focused  on  accomplishing  a  particular, 
narrowly-defined  function.  Generalized,  multi-mission  systems  require  many  more 
components  and  interactions  in  order  to  accomplish  a  wider  range  of  functions,  which 
increases  their  overall  complexity. 

Building  a  basic  system  which  can  be  subsequently  modified  into  multiple 
variants  for  specialized  missions  results  in  less  complexity  than  building  a  single  system 
that  attempts  to  perform  a  wide  range  of  diverse  missions.  Thus,  the  C-130  airframe, 
which  has  40  specialized  variants,  gets  a  high  S-score,  while  the  multi-role  F-22A’s  S- 
score  is  diminished  by  the  bewilderingly  large  number  of  components  necessary  to 
accomplish  all  its  assigned  missions  (Air  Superiority,  Close  Air  Support,  Intelligence, 
Surveillance,  Reconnaissance,  etc.)  with  a  single  aircraft. 

Spinney  points  out  “Increasing  complexity  is  a  cost ...”  (Spinney,  1985:6, 
emphasis  in  original),  so  the  same  simplifying  behaviors  and  decisions  which  improve  a 
system’s  S-score  might  also  improve  the  I-score.  Spinney  also  asserts  that  “the  benefits  of 
increasing  complexity  are  clearly  neither  self-evident  nor  clear-cut.”  (Spinney  1985:85). 
He  points  out  “reliability  is  more  difficult  to  predict  for  complex  systems  than  for  simple 
systems,  because  complexity  increases  the  uncertainty  surrounding  interactions  between 
components...  Moreover,  as  the  complexity  of  a  weapon  increases,  its  number  of  failure 
modes  increases.”  (Spinney,  1985:91).  He  goes  on  to  ask  whether  increasing  a  system’s 
complexity  increases  or  decreases  its  capability.  His  conclusion  is  that  “Increasing 
weapons  complexity  reduces  combat  readiness”  (Spinney,  1985:11)  in  the  following 
ways: 
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•  Degrades  combat  skills  by  causing  inadequate  and  unrealistic  training 

•  Increases  reliability  and  maintainability  problems 

•  Increases  cost  of  maintenance 

•  Increases  dependence  on  large  vulnerable  support  base 

•  Increase  economic  inefficiency  of  plans 

•  Slows  modernization  by  increasing  development/procurement  lead  times 

•  Multiplies  magnitude  and  likelihood  of  disaster 

•  Increases  vulnerability  to  countenneasures 

•  Cuts  forces,  supplies  and  munitions  to  inadequate  numbers 


Spinney  then  rhetorically  asks  if  the  “distinctive  characteristics  generated  by 
weapons  complexity  compensate  for  these  negative  qualities.”  (Spinney,  1985:11).  He 
points  out  that  “High-complexity  aircraft  and  weapons  give  up  presence  (because  high 
cost  reduces  numbers,  and  supportability  problems  reduce  sortie  rates)  in  favor  of 
lethality — a  quality  whose  value  is  maximized  in  an  attrition  model  [of  warfare].” 
(Spinney,  1985:89).  While  small  numbers  of  complex,  expensive  systems  may  offer 
superior  lethality  and  thus  victory  in  an  attrition  model  of  warfare,  Spinney  points  out  “In 
the  American  Civil  War,  World  War  I  and  World  War  II,  the  winners  had  more  casualties 
than  the  losers.  At  the  level  of  a  war  outcome,  attrition  measures  would  indicate  that  the 
winners  lost  and  the  losers  won.”  (Spinney,  1985:89).  Thus,  he  argues  that  a  force  with 
small  numbers  of  super-lethal,  highly  complex  weapons  is  less  effective  than  a  force  with 
larger  numbers  of  less  expensive,  less  complex,  less  lethal  weapons. 

Building  on  Col  John  Boyd’s  famous  “Patterns  of  Conflict”  study,  Spinney 
concludes  “ decreasing  complexity  in  order  to  diminish  friction  and  ‘free  up’  operations 
provides  the  opportunity  to  magnify  the  enemy’s  friction  and  impede  enemy  operations.” 
Thus,  simplicity  not  only  reduces  cost  but  also  creates  the  possibility  of  increasing 
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friction  for  the  opponent.  These  are  all  benefits  which  might  be  cited  by  program 
managers  and  leaders  who  hold  the  Simplicity  value. 


T:  TINY  IS  GOOD 

Finally,  the  T  in  FIST  stands  for  Tiny.  Unlike  the  previous  three  values.  Tiny  is 
seldom  invoked  explicitly,  at  least  not  using  that  particular  tenn.  However,  Tiny  is  often 
practiced  and  pursued  under  the  labels  like  lean,  streamlined,  trim,  agile  or  efficient. 
Organizations  express  this  value  by  fielding  small  teams,  using  miniaturized  components, 
building  small  systems  or  requiring  minimal  documentation,  among  other  behaviors. 
When  Tiny  is  not  valued,  project  leaders  assemble  large  teams  and  large  systems,  judging 
the  size  as  a  measure  of  merit. 

This  piece  of  the  FIST  value  set  was  introduced  in  an  article  titled  “Fist,  Part  5” 

(Quaid  &  Ward,  2006a).  The  article  explains: 

Tiny  is  basically  the  inescapable  outcome  of  the  three  previous  values.  If  your 
project  is  Inexpensive,  it  has  a  Tiny  budget.  If  it  is  Fast,  it  has  a  Tiny  schedule.  A 
Simple  project  has  a  Tiny  degree  of  complexity.  Further,  a  Fast,  Inexpensive,  and 
Simple  project  necessitates  a  Tiny  program  office.  (Quaid  &  Ward,  2006a: 3 1) 

Because  of  the  current  reward  and  recognition  structures  in  the  DoD  today,  the 

Tiny  value  can  be  particularly  difficult  to  implement.  Quaid  &  Ward  illustrate  this 

difficulty  when  they  ask: 

How  can  we  reward  smallness  when  the  most  prestigious  programs  a  program 
manager  can  lead  are  those  with  enormous  budgets,  endless  schedules,  extreme 
complexity,  and  massive  teams?  How  can  we  reward  smallness  when  a  PM’s 
career  path  is  supposed  to  be  one  of  increasing  responsibility,  defined  as  dollars 
and  people  managed?  (Quaid  &  Ward,  2006b:32). 

Thus,  while  Tiny  can  be  understood  as  the  cornerstone  concept  behind  the  FIST 
value  set,  it  is  perhaps  the  most  difficult  to  put  into  practice  and  the  most  rare  to  find. 
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This  is  particularly  true  in  environments  like  the  traditional  NASA  or  DoD  system 
development  communities,  which  tend  to  reward  and  encourage  largeness. 

However,  given  the  prominence  of  Lean  development  approaches,  Agile 
Acquisitions  and  Extreme  Programming  methods,  the  Tiny  value  is  becoming  more 
popular  and  common,  under  a  variety  of  labels.  In  the  1990’s,  NASA’s  Faster,  Better, 
Cheaper  initiative  deliberately  used  small  teams  to  build  small  systems,  imitating  the 
success  of  earlier  small  teams  such  as  Lockheed’s  Skunkworks.  McCurdy  points  out  that 
“small  teams  are  less  expensive  to  staff  than  large  ones.  So  long  as  a  project  remains 
simple,  small  teams  also  possess  an  advantage  in  fighting  failure,”  because,  as  he 
explains,  “Small  teams  solve  problems  informally,  a  process  that  becomes  more  difficult 
as  the  team  grows  in  size.”  (McCurdy,  2001:153).  However,  a  small  team’s  ability  to 
solve  problems  is  degraded  by  complexity,  because  as  complexity  increases,  “it  quickly 
outstrips  the  ability  of  small  teams  to  manage  it.”  (McCurdy,  2001 : 153) 

McCurdy  goes  on  to  praise  the  virtues  of  smallness,  writing  “Smaller  spacecraft 
cost  less  to  construct.  Reliability  problems  on  small  spacecraft  can  be  resolved  through 
teamwork,  which  reduces  both  the  scale  and  the  cost  of  management.”  (McCurdy, 
2001:21).  Thus  we  see  the  strong  interrelations  between  Tiny,  Simple  and  Inexpensive. 

Acquisition  Refonn  efforts  over  the  years  often  tried  to  shrink  the  documentation 
burden  on  project  leaders,  as  when  the  25  page  Lightweight  Fighter  Request  For  Proposal 
limited  contractor  proposals  to  50  pages  (Burton,  1993:19).  The  end  result  of  that  effort  is 
now  known  as  the  F-16  Fighting  Falcon.  Perhaps  the  most  explicit  expressions  of  the 
Tiny  value  are  found  in  the  story  of  the  F-16’s  development,  as  when  the  Falcon’s 
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designer  Harry  Hillaker  said  “People  think  that  big  is  better.  It’s  not.”  (Hehs,  1991). 
Statements  of  value  do  not  get  much  clearer  or  emphatic  than  that. 

Defining  Project  Success 

Since  this  research  aims  to  establish  a  relationship  between  the  FIST  values  and 
successful  system  development,  I  need  a  clear  definition  of  project  success.  However,  as 
Shenhar,  Dvir,  Levy  and  Maltz  pointed  out,  “there  is  very  little  agreement  in  previous 
studies  as  to  what  really  constitutes  project  success.  Traditionally,  projects  were 
perceived  as  successful  when  they  met  time,  budget  and  performance  goals.  However, 
many  would  agree  that  there  is  more  to  project  success  than  meeting  time  and  budget.” 
(Shenhar,  Dvir,  Levy,  Maltz,  2002:699) 

Rather  than  focus  simply  on  programmatic  success  (i.e.  on-time,  on-budget 
delivery  of  a  system  that  satisfies  the  documented  requirements)  or  technological  success 
(i.e.  advancing  the  state  of  the  art),  this  research  examines  the  operational  outcome  of 
system  development  projects.  Specifically,  success  is  defined  as  the  degree  to  which  the 
system  achieved/surpassed  its  functional  objectives,  met/exceeded  its  operational 
requirements,  was  accepted  by  the  user  community  and  generally  changed  the  world. 
This  is  consistent  with  Suddarth’s  observation  that  “For  system  builders,  results  come  in 
the  fonn  of  successfully  used  products.”  (Suddarth,  2002:7). 

Ironically,  whether  or  not  the  system  was  delivered  on  time  or  on  budget  is  not 
necessarily  an  indication  of  a  successful  program,  even  though  budget  and  schedule  are 
typically  the  metrics  used  to  assess  a  program  manager’s  perfonnance  (yet  another  reason 
this  research  uses  the  term  project  leader  instead  of  program  manager).  However,  as  the 
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Air  Force  Studies  Board  explained  in  their  assessment  of  the  relationship  between 

systems  engineering  and  defense  acquisition  program  outcomes: 

...  programs  that  succeeded  were  those  that  delivered  their  products  within  a 
reasonable  margin  of  the  original  cost  and  schedule  baseline.  Programs  that  failed, 
in  the  committees  view,  may  have  delivered  successful  products  but  were  well 
outside  the  reasonable  expectations  of  the  original  program  and  were  only 
successful  in  delivering  products  after  the  addition  of  substantial  unplanned 
funding  and  a  substantial  extension  of  the  original  schedule.  (Air  Force  Studies 
Board,  2008:22). 

The  board’s  opinion  is  primarily  focused  on  the  concept  of  programmatic  instead 
of  operational  success,  but  nonetheless  delivering  “within  a  reasonable  margin  of  the 
original  cost  and  schedule  baseline”  is  one  of  the  factors  taken  into  account  in  this 
research  when  assessing  the  program’s  outcome.  While  delivering  on  time  and  on  budget 
does  not  necessarily  ensure  operational  success,  failure  to  do  so  can  have  a  strong  affect 
on  the  system’s  operational  outcome.  If  a  system  is  delivered  too  late  to  be  of  use  (for 
example,  after  the  war  is  over),  its  operational  impact  on  that  particular  conflict  is 
obviously  zero.  Similarly,  if  the  system’s  cost  rises  to  the  point  that  it  is  not  affordable  in 
sufficient  quantity,  its  operational  effectiveness  is  diminished. 

Some  operational  outcomes  are  clearer  cut  than  others.  For  example,  NASA’s  X- 
Ray  Timing  Explorer’s  mission  was  described  this  way:  “The  spacecraft  was  developed, 
tested  and  ready  for  launch  several  months  early,  and  was  completed  significantly  under 
the  fixed-price  budget.  It  was  successfully  launched  and  has  met  all  mission  objectives.  It 
is  considered  a  dreamboat  by  the  scientific  investigators,  who  have  made  many  important 
discoveries.”  (Laufer,  2000:37).  This  is  clearly  an  example  of  an  unequivocally  positive 
outcome,  both  operationally  and  programmatically. 


26 


Another  NASA  project  was  described  this  way  “other  project  managers  saw  that 
our  experiment  worked,  learned  from  it,  and  adopted  the  approach  for  their  projects...” 
(Laufer,  2000:45)  Imitation  by  one’s  peers  is  also  strong  evidence  of  a  positive 
operational  outcome.  However,  when  a  story  simply  concludes  “we  delivered  on  time,” 
this  is  weak  evidence  of  positive  operational  outcome,  and  by  omitting  any  reference  to 
the  customer’s  reaction  or  use  of  the  system,  may  even  be  evidence  that  the  system  was 
less  than  successful  in  actual  operational  use. 

Focusing  on  operational  rather  than  simply  programmatic  outcomes  means 
focusing  on  the  customer,  rather  than  the  program  manager.  This  is  an  important 
distinction,  often  missing  from  the  systems  engineering  and  program  management 
literature.  I  chose  to  emphasize  customer  success  rather  than  the  program  manager’s 
success,  because  delivering  effective  systems  is  supposed  to  be  the  ultimate  objective  of 
any  system  development  project,  particularly  those  done  by  the  DoD. 

In  the  data  analysis  portion  of  this  research,  the  project  or  organization’s 
operational  outcome  is  compared  with  their  FIST  score.  All  the  scores  and  outcomes  are 
compiled  to  determine  a  correlation  between  the  FIST  value  set  and  operational  outcomes. 
In  the  conclusion,  the  stories  and  insights  are  consolidated  into  heuristics,  following  the 
example  of  Maier  and  Rechtin’s  The  Art  of  Systems  Architecting. 

A  Note  on  Trust  and  Talent 

System  development  program  outcomes  are  determined  by  the  people  involved. 
Successful  program  managers  go  out  of  their  way  to  instead  unleash  their  team’s  talent 
rather  than  dictate  and  control  individual  behavior.  Very  talented  people  can  provide 
positive  results  despite  bad  or  inefficient  processes,  but  the  best  process  in  the  world 
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cannot  fully  compensate  for  a  lack  of  talented  people.  As  Laufer  explains,  “To  cope  with 
conditions  of  very  high  speed,  coupled  with  very  high  technological  uncertainty,  you 
must  staff  your  project  with  highly  skilled,  dedicated  and  resourceful  people.”  (Laufer, 
2000:  81). 

Many  other  writers  and  researchers  have  made  and  tested  these  claims,  arriving  at 
similar  conclusions.  A  recent  and  prominent  example  is  the  Air  Force  Research  Board, 
which  highlighted  “six  seeds  of  failure.”  In  assessing  these  seeds,  the  Board’s  report 
concluded  ”...  the  biggest  risk  of  all  in  undertaking  large  development  programs  is  to 
proceed  with  less  than  the  best  personnel”  (Air  Force  Studies  Board,  2008:67).  Therefore, 
this  research  takes  it  as  axiomatic  that  organizational  success,  both  operational  and 
programmatic,  requires  trust  and  talent  and  is  primarily  determined  by  the  people 
involved.  The  referenced  literature  has  sufficiently  proven  this  assertion,  and  no  attempt 
is  made  in  this  thesis  to  re-prove  the  idea  that  talent  is  vitally  important  and  people  are  an 
organization’s  most  important  asset.  In  fact,  the  primacy  of  people  is  precisely  what 
makes  the  question  of  values  so  crucial. 

FIST  and  Systems  Engineering 

Systems  engineering  is  concerned  with  both  technology  and  organization,  the 
mathematical  and  the  social.  In  a  sense,  Systems  engineering  is  the  intersection  of 
complex  technology  and  complex  organization.  Thus,  this  research  applies  systems 
thinking  and  looks  at  FIST  as  both  an  organizational  value  set  and  a  design  value  set, 
applicable  to  the  entities  which  design  technology  projects  as  well  as  the  technology 
development  projects  themselves. 
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Systems  engineers  and  technology  leaders  sometimes  give  the  impression  that 
given  enough  time  and  money,  they  can  do  anything.  This  does  not  appear  to  be  true.  The 
Comanche  helicopter  project  spent  21  years  and  $7B  before  it  was  cancelled,  having 
delivered  zero  aircraft.  It  is  doubtful  another  decade  and  a  few  billion  more  dollars  would 
have  improved  the  outcome. 

This  research  deconstructs  and  examines  the  “we  can  do  anything  given  enough  time 
and  money”  component  of  the  systems  engineering  mythos.  It  suggests  that  operational 
success,  in  which  systems  are  available  when  needed  and  effective  when  used,  is  not 
necessarily  ensured  by  large  budgets  and  long  schedules.  Indeed,  the  data  shows  that 
excessive  amounts  of  time  and  money,  coupled  with  a  preference  for  complexity,  can 
lead  directly  to  operational  failure.  In  contrast,  small  budgets  and  short  timelines  seem  to 
have  a  simulative  effect  and  actually  support  the  goal  of  delivering  an  operational  system 
that  is  available  when  needed  and  effective  when  used. 
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II. 


Literature  Review 


Analyses  of  the  defense  acquisition  community  or  NASA’s  mission  development 
efforts  are  not  hard  to  find.  For  several  decades,  critics  of  these  two  groups  have  echoed 
some  variation  of  the  theme  that  technology  development  efforts  are  too  expensive,  take 
too  long,  are  too  complicated  and  too  big.  It  is  not  a  difficult  case  to  make. 

The  ubiquitous  refrain  points  to  a  traditional  value  system  within  NASA  and  the 
DoD  that  rewards  large  budgets,  long  timelines,  complex  systems  and  enonnous 
bureaucracies...  which  leads  to  programs  that  overrun  and  under  deliver.  Many  of  these 
critics  offer  some  variation  of  the  FIST  value  set  as  an  alternative  to  the  current 
environment,  while  others  call  for  more  oversight,  more  fonnal  processes,  and  stricter 
bureaucratic  or  legislative  control. 

The  following  books,  reports  and  articles  are  a  brief  sample  of  some  of  the  most 
prominent  and  insightful  assessments  of  DoD  and  NASA  system  development  programs. 
The  majority  of  the  reports  are  contemporary,  but  several  are  historical,  indicating  that 
the  FIST  value  set  is  not  a  new  discovery. 

Assessments  of  Selected  Weapon  Programs  (GAO-08-467SP) 

In  its  sixth  annual  assessment  of  selected  DoD  weapon  programs,  the  GAO 

bluntly  states  “cost  and  schedule  outcomes  for  major  weapon  programs  are  not  improving 
over  the  6  years  we  have  been  issuing  this  report”  (Dodaro,  2008:1),  even  though 
“DOD’s  planned  investment  for  new  weapon  systems  now  reflects  the  highest  funding 
levels  in  two  decades.”  (Dodaro,  2008:3). 
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The  report  highlights  disturbing  trends  in  development  costs  and  timelines,  and 
consistently  provides  explicit  support  to  many  of  the  FIST  values.  For  starters,  it  points 
out  “A  hallmark  of  an  executable  program  with  a  sound  business  case  is  short 
development  cycle  times.”  (Dodaro,  2008:26)  In  other  words,  the  GAO  affirms  that  it  is 
important  and  good  to  be  fast,  supporting  the  Fast  value. 

The  report  goes  on  to  explain  “Long  cycle  times  promote  instability,  especially 
considering  DOD’s  tendency  to  have  changing  requirements  and  program  manager 
turnover.  In  fact,  DOD  itself  suggests  that  system  development  should  be  limited  to  about 
5  years.  Time-defined  constraints  such  as  this  are  important  because  they  serve  to  limit 
the  initial  product’s  requirements,  allow  for  more  frequent  assimilation  of  new 
technologies  into  weapon  systems,  and  speed  new  capabilities  to  the  warfighter.” 
(Dodaro,  2008:26).  [ emphasis  added]. 

The  report  is  critical  of  DoD  perfonnance  in  the  area  of  schedule,  pointing  out 
that  “on  average,  the  current  portfolio  of  programs  has  experienced  a  21 -month  delay  in 
delivering  initial  operational  capability  to  the  warfighter.”  (Dodaro,  2008:8).  Clearly,  the 
GAO  deems  it  important  and  good  for  development  programs  to  be  faster  than  they  are, 
and  is  disappointed  by  the  current  pace  of  acquisitions. 

On  the  topic  of  costs,  the  report  states  that  “development  costs  for  fiscal  year 
2007  programs  increased  by  40  percent  from  first  estimates,  compared  to  27  percent  for 
fiscal  year  2000  programs.”  (Dodaro,  2008:4).  It  goes  on  to  say  “Total  acquisition  cost 
for  the  current  portfolio  of  major  programs  under  development  or  in  production  has 
grown  by  nearly  $300  billion  over  initial  estimates.”  (Dodaro,  2008:8).  Either  the  DoD  is 
getting  worse  at  estimation  or  we  are  getting  worse  at  controlling  costs.  In  either  case,  the 
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cost  growth  rate  is  not  good.  This  observation  supports  the  Inexpensive  value  within  the 
FIST  value  set. 

As  for  simplicity,  the  GAO  reports  “Ninety-six  percent  of  the  programs  had  not 

met  best  practice  standards  for  demonstrating  mature  technologies  and  design  stability 

before  entering  the  more  costly  system  demonstration  phase.”  (Dodaro,  2008:4)  Use  of 

mature  technologies  and  stable  designs  is  a  critical  component  of  mature  simplicity,  as 

defined  in  The  Simplicity  Cycle,  so  this  criticism  supports  the  Simple  value. 

Complexity  (along  with  its  companion  size)  is  frequently  cited  as  a  risk-driver  in 

the  weapon  systems  under  evaluation,  as  in  the  following  three  examples: 

...development  of  MUOS  ground  software  represents  one  of  the  highest  risks  to 
the  program  due  to  the  size  and  complexity  of  the  contractor’s  design.  (Dodaro, 
2008:144) 

The  complexity  of  the  GEO  satellites  is  greater  than  that  of  the  HEO  sensors,  and 
as  of  September  2007,  only  20  percent  of  planned  integration  testing  on  the  first 
satellite  was  complete.  As  such,  there  is  high  probability  that  further  design  flaws 
may  be  discovered,  leading  to  more  cost  and  schedule  increases.  (Dodaro, 
2008:154) 

According  to  program  officials,  the  seeker  will  be  the  most  challenging 
technology  to  demonstrate  due  to  the  complexity  of  the  algorithms  it  will 
require...  (Dodaro,  2008:156) 

The  report  concludes  that  spending  more  money  and  more  time  on  development  is 
not  improving  acquisition  outcomes.  The  GAO  writes  “Our  analysis  does  not  show  any 
improvements  in  acquisition  outcomes  as  programs  continue  to  experience  increased 
costs  and  delays  in  delivering  capabilities  to  the  warfighter.  In  fact,  when  compared  to  the 
perfonnance  of  the  fiscal  year  2000  portfolio  of  major  defense  acquisition  programs,  cost 
and  schedule  performance  for  current  programs  is  actually  worse.”  (Dodaro,  2008:6) 
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While  the  report  does  not  assess  the  size  of  program  offices  or  the  complexity  of 
required  processes,  it  can  reasonably  be  inferred  that  these  Major  Defense  Acquisition 
Programs  are  indeed  large,  not  Tiny. 

Better  Weapon  Program  Outcomes  Require  Discipline,  Accountability,  and 
Fundamental  Changes  in  the  Acquisition  Environment  (GAO-08-782T) 

This  GAO  report  documents  the  testimony  of  Katherine  Schinasi,  Managing 

Director  of  Acquisition  and  Sourcing  Management.  It  summarizes  and  draws  on  the 
GAO’s  extensive  “body  of  work  on  DOD’s  acquisition,  requirements  and  funding 
processes,  as  well  as  our  annual  assessment  of  selected  DOD  weapon  programs.” 
(Schinasi,  2008:1).  Thus,  it  echoes  the  observations  and  recommendations  of  the 
previously  cited  GAO  report  (Dodaro,  2008). 

After  pointing  out  that  “Congress  and  DOD  have  continually  explored  ways  to 
improve  acquisition  outcomes  without  much  to  show  for  their  efforts,”  (Schinasi,  2008:1), 
the  report  goes  on  to  make  several  statements  bemoaning  the  fact  that  the  DoD  is  not 
implementing  the  FIST  values.  Many  of  the  report’s  statements  supporting  the  FIST 
values  are  presented  in  the  negative,  criticizing  the  absence  of  these  values  rather  than 
advocating  their  presence,  but  the  overall  effect  is  to  provide  support  to  the  importance 
and  goodness  of  the  FIST  values. 

Specifically,  the  report  states  “the  requirements  and  acquisition  processes  are  not 
agile  enough  to  support  programs  that  can  meet  current  operational  requirements.” 
(Schinasi,  2008:2)  It  is  not  unreasonable  to  read  this  as  a  statement  that  the  processes  are 
too  big  and  too  complex.  Smaller,  simpler  processes  would  likely  provide  a  greater 
degree  of  agility  than  that  found  in  today’s  environment. 
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The  report  goes  on  to  state  “the  acquisition  environment  does  not  provide 

appropriate  incentives  for  contractors  to  stay  within  cost  and  schedule  targets.”  (Schinasi, 

2008:2)  By  not  incentivizing  speed  and  thrift,  the  acquisition  environment  does  not  hold 

to  the  Fast  and  Inexpensive  values.  The  clear  implication  is  that  acquisition  outcomes 

would  be  improved  if  the  environment  supported  the  idea  that  it  is  important  and  good  to 

be  Fast  and  Inexpensive,  and  rewarded  those  behaviors  accordingly. 

In  an  earlier  testimony  to  the  US  Senate,  Schinasi  clearly  highlighted  the  negative 

impact  of  FIST’s  absence:  “In  an  important  sense,  success  [in  DoD  acquisitions]  has 

come  to  mean  starting  and  continuing  programs  even  when  cost,  schedule,  and  quantities 

must  be  sacrificed.”  (Schinasi,  2005:4)  In  other  words,  it  is  apparently  not  important  to  be 

Fast  or  Inexpensive,  a  situation  Schinasi  deems  unfortunate  and  inappropriate. 

The  report  points  out  the  negative  impact  caused  by  a  failure  to  implement  the 

Fast  and  Inexpensive  values:  “Continued  cost  growth  results  in  less  funding  being 

available  for  other  DOD  priorities  and  programs,  while  continued  failure  to  deliver 

weapon  systems  on  time  delays  providing  critical  capabilities  to  the  warfighter.” 

(Schinasi,  2008:5)  These  are  obvious  points,  but  apparently  they  need  to  be  stated.  The 

report  goes  on  to  explain  the  importance  and  benefits  of  being  Fast: 

Constraining  development  cycles  would  make  it  easier  to  more  accurately 
estimate  costs,  and  as  a  result,  predict  the  future  funding  needs  and  effectively 
allocate  resources.  We  have  consistently  emphasized  the  need  for  DOD’s 
weapon  programs  to  establish  shorter  development  cycles.  DOD’s 
conventional  acquisition  process  often  requires  as  many  as  10  or  15  years  to  get 
from  program  start  to  production.  Such  lengthy  cycle  times  promote  program 
funding  instability — especially  when  considering  DOD’s  tendency  to  change 
requirements  and  funding  as  well  as  frequent  changes  in  leadership. 

Constraining  cycle  times  to  5  or  6  years  would  force  programs  to  conduct  more 
detailed  systems  engineering  analyses,  lend  itself  to  fully  funding  programs  to 
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completion,  and  thereby  increase  the  likelihood  that  their  requirements  can  be  met 
within  established  time  frames  and  available  resources.  An  assessment  of  DOD’s 
acquisition  system  commissioned  by  the  Deputy  Secretary  of  Defense  in  2006 
similarly  found  that  programs  should  be  time-constrained  to  reduce  pressure  on 
investment  accounts  and  increase  funding  stability  for  all  programs.  (Schinasi, 
2008:9)  [emphasis  added] 


The  report  concludes  with  an  observation  that  changes  to  the  DoD’s 
organizational  values  -  the  statements  of  priorities  and  preferences,  the  artifacts  which 
are  deemed  important  and  good  and  are  therefore  rewarded  -  must  be  at  the  heart  of  any 
effective  reform  efforts:  “Meaningful  and  lasting  refonn  will  not  be  achieved  until  DOD 
changes  the  acquisition  environment  and  the  incentives  that  drive  the  behavior  of  DOD 
decision  makers,  the  military  services,  program  managers  and  the  defense  industry.” 
(Schinasi,  2008:12-13). 

Air  Force  Studies  Board  Report:  Pre-Milestone  A  and  Early  Phase  Systems 
Engineering,  2008 

This  book  by  the  Air  Force  Studies  Board  compares  the  historical  timelines  and 

costs  of  system  development  with  current  performance,  observing 

The  time  required  to  execute  large,  government-sponsored  systems  development 
programs  has  more  than  doubled  over  the  past  30  years,  and  the  cost  growth  has 
been  at  least  as  great.  Many  causes  for  this  trend  have  been  suggested,  including 
the  increased  complexity  of  the  systems  involved;  instability  of  external 
funding...  (AF  Studies  Board,  2008:14). 

By  identifying  complexity  as  a  potential  cause,  the  Board  is  perhaps  suggesting 
the  Simple  value  would  help  improve  acquisition  outcomes.  Indeed,  the  Board  goes  on  to 
call  out  several  specific  programs,  again  returning  to  the  idea  that  complexity  contributes 
to  explosive  growth  in  costs  and  delivery  times. 
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During  World  War  II  and  the  early  Cold  War,  programs  such  as  the  Manhattan 
Project,  the  Defense  Support  Program  (DSP),  the  intercontinental  ballistic  missile, 
and  the  U-2  surveillance  aircraft  all  delivered  very  quickly,  generally  in  fewer 
than  6  years,  first  products  that  today  would  be  described  as  major  systems. 
Currently,  such  major  programs  would  likely  require  10  to  20  years  to  complete. 
System  complexity  has  grown  dramatically,  and  products  are  not  delivered  under 
the  same  technological,  human,  and  organizational  guidelines  as  before.  There  has 
been  about  a  threefold  increase  in  delivery  time  for  most  major  systems.  (AF 
Studies  Board,  2008:14) 

The  Board  identifies  “the  six  seeds  of  failure,”  five  of  which  are  related  to 
complexity.  The  only  seed  not  related  to  complexity  is  “inexperienced  leadership,”  a 
concept  that  resonates  with  the  FIST  principle  of  the  importance  of  talent. 

The  remaining  five  complexity-related  seeds  are  “external  interface  complexity; 
system  complexity;  incomplete  or  unstable  requirements  at  Milestone  B;  reliance  on 
immature  technology;  and  reliance  on  large  amounts  of  new  software.”  (AF  Studies 
Board,  2008:67).  This  list  could  be  simplified  by  consolidating  “interface  complexity” 
and  “system  complexity”  into  a  single  entity  called  “complexity.”  Similarly,  the 
distinction  between  “immature  technology”  and  “new  software”  is  hardly  insightful  - 
new  software  is,  by  definition,  immature.  Requirements  instability  is  often  the  result  of 
confusion  caused  by  excessive  complexity.  As  project  leaders  become  more  familiar  with 
the  system’s  complexities,  they  adjust  the  requirements  accordingly.  Simplicity  tends  to 
act  as  a  stabilizing  force  on  requirements  (as  does  speed). 

The  solution  to  this  unfortunate  trend  towards  increased  complexity  has  largely 
been  to  implement  complex  “process  improvement”  efforts.  However,  the  Board  points 
out: 


...  despite  the  myriad  of  new  and  revised  processes  throughout  government 
acquisition  organizations,  there  is  little  sign  that  performance  is  returning  to  the 
development  productivity  that  was  achieved  decades  ago.  Indeed,  one  is  tempted 
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to  conclude  that  performance  diminishes  as  procurement  organizations  mature 
and  their  processes  become  more  complex.  (AF  Studies  Board,  2008: 16) 

Again,  the  theme  of  complexity  overwhelming  effectiveness  is  repeated 

throughout  the  document.  This  supports  the  idea  that  organizational  simplicity,  properly 

understood,  is  conducive  to  effective  system  development  projects. 

The  report  also  offers  an  interesting  perspective  on  determining  whether  a 

program  is  successful  or  not. 

Programs  that  failed,  in  the  committees  view,  may  have  delivered  successful 
products  but  were  well  outside  the  reasonable  expectations  of  the  original 
program  and  were  only  successful  in  delivering  products  after  the  addition  of 
substantial  unplanned  funding  and  a  substantial  extension  of  the  original  schedule. 
(AF  Studies  Board,  2008:26) 

This  observation  has  interesting  implications  for  projects  like  the  F-22  and  V-22, 
both  of  which  were  only  successful  “after  the  addition  of  substantial  unplanned  funding” 
and  “substantial  extension[s]  of  the  original  schedule.”  The  Board  seems  to  be  saying  that 
such  aircraft  should  be  viewed  as  failures. 

The  Board  concludes  with  a  number  of  recommendations  which  are  largely 
consistent  with  the  FIST  values.  Along  with  encouraging  the  pursuit  of  simplicity,  the 
Board  explicitly  supports  the  Fast  value.  They  write  “...  the  world  changes  too  fast  to  be 
friendly  to  long  development  cycles.  The  committee  believes  that  the  Air  Force  should 
strive  to  structure  major  development  programs  so  that  initial  deployment  is  achieved 
within,  say  3  to  7  years.”  (AF  Studies  Board,  2008:85) 

Better,  Faster,  Cheaper,  2001 

In  1992,  NASA  administrator  Daniel  Goldin  launched  the  agency’s  “faster,  better, 
cheaper”  (FBC)  initiative.  The  FBC  initiative  was  built  on  many  of  the  FIST  values,  to 
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include  simplicity  and  smallness,  both  in  terms  of  the  technology  being  developed  and 

the  organizations  doing  the  development.  The  approach  of  FBC  project  leaders  was  quite 

different  than  the  traditional  NASA  approach,  as  McCurdy  explains: 

The  people  who  manage  “faster,  better,  cheaper”  projects  have  exactly  the  same 
objectives:  prevent  mission  failure  while  controlling  cost  and  schedule.  They 
attempt  to  do  this,  however,  in  a  fundamentally  different  way.  They  cannot  rely 
upon  elaborate  systems  management  procedures,  which  are  too  expensive  and 
time-consuming...  Instead,  people  who  run  such  missions  turn  to  the  dynamics 
that  arise  in  small,  cohesive  project  teams.  (McCurdy,  2001:89) 

McCurdy  identifies  15  techniques  used  by  FBC  project  leaders.  Many  of  these 

techniques  highlight  the  organizational  values  used  by  NASA  on  FBC  projects,  helping 

to  identify  and  define  several  measures  of  merit.  These  techniques  including  the 

following: 

•  Cost  goals.  Project  leaders  accept  cost  containment  (and  schedule  control) 
as  a  major  goal. 

•  Project  scale.  The  spacecraft  is  small  and  the  project  team  that  develops 
and  flies  it  is  very  small. 

•  Cancellation.  Top  executives  are  prepared  to  cancel  any  project  where  cost 
or  schedule  spiral  out  of  control,  and  team  members  are  told  that  this  will 
happen. 

•  Risk-taking.  Team  members  are  encouraged  to  be  creative  and  take 
calculated  risks  (but  told  not  to  fail).  (McCurdy,  2001:24-25) 


There  were  a  total  of  16  FBC  projects  over  the  next  8  years,  of  which  6  failed. 
This  63%  success  rate  was  deemed  unacceptable,  and  the  fonnal  FBC  initiative  was 
abandoned  in  1999  (McCurdy,  2001),  despite  the  initial  appreciation  that  FBC  projects 
are  higher  risk  and  therefore  had  a  lower  expected  success  rate.  Interestingly,  following 
the  $813M  Mars  Observer  failure,  NASA  did  not  reject  the  traditional  approach  or 
replace  their  value  set,  nor  did  they  do  so  after  any  number  of  other  costly,  high-profile 
failures.  Apparently  the  consensus  among  NASA  leaders  was  that  they  had  spent  so  much 
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money  on  those  failed  projects  that  the  failure  must  have  an  external  cause,  and  therefore 
internal  changes  were  not  needed.  The  data  suggests,  however,  that  the  excessive 
spending  rates  and  the  accompanying  complexity,  both  organizational  and  technical,  may 
have  been  a  contributing  factor  in  the  failures. 

Despite  the  lower  than  expected  success  rate  across  the  cohort,  the  Faster,  Better, 
Cheaper  approach  still  has  a  significant  number  of  proponents  and  offers  some  interesting 
insights  into  the  system  development  process.  Supporters  point  out  that  the  total  cost  for 
the  16  FBC  projects  was  less  than  the  cost  of  one  large  planetary  expedition.  For  example, 
the  Cassini  mission  to  Saturn  cost  more,  in  inflation- adjusted  dollars,  than  all  16  FBC 
projects,  and  the  16  projects  included  “five  missions  to  Mars,  one  mission  to  the  moon, 
three  space  telescopes,  two  comet  and  asteroid  rendezvous,  four  Earth-orbiting  satellites, 
and  one  ion  propulsion  test  vehicle.”  (McCurdy,  2001 : 19). 

It  is  also  worth  considering  that  nine  of  the  first  ten  FBC  projects  were  successes, 
and  four  of  the  six  failures  occurred  in  1999.  (McCurdy,  2001:2).  Perhaps  something  was 
happing  in  1999  to  cause  these  failures,  unrelated  to  the  FBC  approach  itself. 

After  studying  the  entire  cohort  of  FBC  missions,  McCurdy  reached  the  following 
conclusion:  “Engineers  and  other  experts  can  reduce  the  cost  of  spaceflight  and  the  time 
necessary  to  prepare  missions  for  flight.  Moreover,  they  can  do  so  without  significant 
loss  of  reliability.  They  can  also  do  so  with  only  modest  reductions  in  spacecraft 
capability.”  (McCurdy,  2001:10).  Alongside  the  focus  on  simplicity,  the  willingness  to 
make  modest  reductions  in  capability  is  a  key  aspect  of  FBC. 

It  is  important  to  recognize  that  “better”  is  a  subjective  assessment,  and  according 
to  McCurdy,  a  “reduced  capability  does  not  mean  the  mission  is  automatically  worse.  A 
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mission  with  one-half  the  capability  will  be  ‘better’  if  it  performs  that  mission  at  one- 
tenth  the  price.”  (McCurdy,  2001:22).  This  is  a  philosophical  position,  an  expression  of 
the  organization’s  values.  It  is  based  on  the  organization’s  answer  to  the  question  “What 
is  good?”  Under  FBC,  a  system  with  reduced  capabilities  was  still  considered  good  if  it 
delivered  those  capabilities  under  drastically  reduced  budgets  and  timelines.  In  other 
words,  FBC  was  a  FISTy  initiative. 

It  appears  that  the  problem  with  FBC  is  not  that  it  is  mathematically  correct  but 
operationally  incorrect,  or  “good  in  theory,  bad  in  practice.”  Rather,  it  is  a  scary  theory 
and  requires  project  leaders  to  courageously  embrace  values  that  run  counter  to  those 
held  by  the  defenders  of  the  status  quo.  The  FBC  approach  is  therefore  not  put  in  practice 
for  reasons  having  nothing  to  do  with  mechanical  ease  and  everything  to  do  with 
psychological  comfort.  The  key  issues  revolve  around  fear,  trust,  control  and 
organizational  values,  not  any  inherent  mathematical,  technical  or  financial  flaw  in  the 
FBC  reasoning. 

McCurdy  points  out  that  “proponents  of  the  approach  created  failure  when  they 
reduced  cost  and  schedule  faster  than  they  lessened  complexity.”  (McCurdy,  2001:25), 
reiterating  the  concept  that  simplicity  is  an  essential  element  in  effective  FBC  technology 
projects.  Attempts  were  often  made  to  fix  a  problem  by  adding  additional  funds,  in  the 
belief  that  additional  spending  on  redundancies  would  increase  reliability.  However, 
McCurdy  points  out  that  “Additional  spending  does  not  always  purchase  added  reliability. 
It  buys  complexity.  Added  complexity  creates  a  point  of  diminishing  returns,  which 
eventually  reduced  reliability.”  (McCurdy,  2001 : 155). 
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McCurdy  also  points  out  that  “increased  spending  does  not  compensate  for 
project  that  managers  developed  too  rapidly,”  and  goes  on  to  observe  that  “a  project 
developed  too  fast  and  too  cheaply  will  almost  certainly  develop  problems  of  reliability, 
but  so  will  one  that  is  simply  too  fast.”  (McCurdy,  2001:27)  His  observations  echo  this 
research’s  previous  comments  about  the  danger  of  speeding. 

After  highlighting  some  of  the  dangers  and  pitfalls  inherent  in  the  FBC  approach, 
McCurdy  concludes: 

The  ‘faster,  better,  cheaper’  approach  can  overcome  problems  of  reliability.  To  do 
so,  however,  requires  close  adherence  to  a  number  of  principles.  The  designing 
team  must  reduce  spacecraft  size  and  complexity  through  micro-technology. 
Supporters  must  fund  the  project  at  a  level  commensurate  with  its  complexity. 
They  must  provide  an  adequate  development  schedule,  again  based  on  mission 
complexity.  And  they  must  rigorously  utilize  team-based  management  techniques. 
(McCurdy,  2001:30) 

McCurdy’s  conclusion  closely  aligns  with  the  FIST  model’s  emphasis  on 
simplicity  and  smallness,  for  both  the  organization  and  the  technology.  Reducing 
complexity  allows  for  shorter  development  schedules  and  smaller  budgets,  while  still 
delivering  the  required  levels  of  reliability  and  perfonnance. 

NASA’s  FBC  initiative  clearly  was  a  value-driven  effort,  an  attempt  to  present  an 
alternative  to  the  agency’s  traditional  preference  for  expensive,  large,  complex,  long-tenn 
missions.  By  defining  low  cost,  rapid  delivery,  simplicity  and  smallness  as  measures  of 
merit,  NASA  explored  new  possibilities  and  demonstrated  that  even  for  the  most  difficult 
mission  areas,  the  FBC  approach  can  deliver.  By  failing  to  explicitly  include  simplicity  in 
that  value  set,  NASA  planted  the  seeds  of  the  initiative’s  failure. 
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On  The  Road  Toward  Confirming  Augustine’s  Predictions  And  How  To  Reverse 
Course,  2007 

This  article  was  published  in  the  Dec  2007  issue  of  Defense  Acquisition  Review 
Journal,  by  Dr.  Jan  Muczyk,  fonner  Dean  of  the  Graduate  School  of  Logistics  and 
Acquisition  Management  at  the  Air  Force  Institute  of  Technology.  Arguing  for  the 
operational  superiority  of  large  quantities  of  simple  weapon  systems  over  small  quantities 
of  complicated  “transformational  technology,”  Muczyk  writes  “quantity  has  its  own 
quality  advantages.  However,  quantity  can  only  be  attained  by  short  product  development 
cycles,  and  that  is  only  achievable  if  the  Department  of  Defense  relies  whenever 
practicable  on  an  evolutionary  approach  utilizing  low-hanging  fruit  and  off-the-shelf 
commercial  components.”  (Muczyk,  2007:455). 

In  other  words,  Muczyk  argues  that  the  DoD  should  adopt  the  Fast,  Inexpensive 
and  Simple  values  (and,  by  implication,  the  Tiny  value  as  well).  The  article  strongly 
argues  for  the  importance  of  rapidly  developing  low-cost  solutions  using  mature 
technology  -  explicitly  affirming  the  F,  I  and  S  portions  of  the  FIST  model. 

On  the  importance  of  speed,  Muczyk  writes  “Perhaps  the  greatest  danger 
associated  with  lengthy  product  development  cycles  is  the  mission  obsolescence  of  the 
weapon  system  before  it’s  even  fielded  because  the  facts  on  the  ground  change  so  fast.” 
(Muczyk  2007:458).  The  article  goes  on  to  explain  “Future  threats  will  be  far  less 
predictable  than  those  during  the  Cold  War  era.  Consequently,  future  DoD  leaders  will 
have  to  name  that  tune  after  hearing  just  a  few  notes,  and  short  cycle  times  will  give  them 
the  ability  to  fashion  appropriate  and  affordable  technological  responses...  we  must 
acquire  dominance  of  product  development  cycle  time  in  order  to  maintain  our 
competitive  edge  on  future  battlefields.”  (Muczyk  2007:461). 
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As  for  the  Inexpensive  value,  Muczyk  offers  very  strong  support  of  the  idea  that 
small  budgets  are  “important  and  good”  -  the  Inexpensive  value.  He  writes  “reducing 
cost  by  reducing  cycle  time  is  critical,”  (Muczyk  2007:461)  and  highlights  the 
interconnectedness  of  the  FIST  values  by  pointing  out  that  “developing  weapon  systems 
from  low-hanging  fruit  pretty  much  guarantees  short,  less  expensive  product 
development  cycles.”  (Muczyk  2007:461)  That  is,  the  pursuit  of  simplicity  tends  to 
reduce  the  amount  of  time  and  money  required  to  field  a  system...  and  these  reductions 
are  desirable. 

Taking  an  approach  similar  to  this  thesis,  Muczyk  examines  the  operational 
effectiveness  of  several  historical  weapon  systems,  and  ultimately  concludes  that  simpler 
systems  outperform  complex  ones  in  actual  combat  operations.  He  examines  the  US 
bomber  inventory  (the  B-1B,  B-2  and  B-52),  and  writes  “of  the  three  bomber  fleets,  given 
contemporary  threats,  the  B-52  is  the  most  cost-effective  to  operate,  and  its  standoff 
weapons  probably  the  most  versatile.”  (Muczyk  2007:456).  The  more  complex  and  more 
expensive  bombers  (B-l  and  B-2)  are  actually  less  effective  than  the  older,  cheaper, 
simpler  B-52. 

Muczyk  continues  his  argument  that  large  quantities  of  simple  systems  have 
historically  defeated  small  cohorts  of  more  complex  weapons,  despite  the  simpler 
systems  offering  inferior  individual  perfonnance.  Looking  back  to  the  World  War  II  era, 
Muczyk  points  out  “The  U.S.  Shennan  tank  was  inferior  in  most  respects  to  the  German 
counterparts,  but  we  prevailed  with  it  because  we  possessed  it  in  massive  numbers. . .  The 
German  Me-262  jet  fighter,  even  with  its  considerable  speed  advantage,  had  little  bearing 
on  the  air  war  because  of  its  limited  numbers.  The  United  States,  however,  prevailed  in 
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the  air  in  Europe  and  in  the  Pacific  because  it  possessed  massive  bomber  and  fighter 
fleets...  In  like  manner,  the  German  advantage  in  rocket  technology  did  little  to  influence 
the  outcome  of  the  war.  The  much  simpler  Soviet  Katyusha  rocket  had  a  much  greater 
impact,  in  part  due  to  its  ubiquitous  presence  on  the  battlefield.”  (Muczyk  2007:460). 

The  article  concludes  with  a  recommendation  that  the  acquisition  community 
embrace  the  Simple  value,  specifically  in  the  area  of  using  mature  technology  rather  than 
relying  on  new,  unproven  developments.  Muczyk  writes  “The  individual  engaged  in  the 
oversight  must  ensure  that  whenever  possible  low-hanging  technological  fruit  and  off- 
the-shelf  components  are  incorporated  into  the  weapon  system.”  This  pursuit  of 
simplicity,  as  previously  stated,  tends  to  reduce  costs  and  timelines,  and  enable  larger 
quantities  to  be  purchased  and  fielded. 

Gregory:  The  Defense  Procurement  Mess,  1989 

This  book  highlights  several  aspects  of  the  military  technology  development 

quandary,  including: 

•  Overcomplex  solutions  (Gregory,  1989:2) 

•  Overmanagement,  overregulation,  overspecification  (Gregory,  1989:  3) 

•  “Massive  paperwork  requirement. . .  costing  from  25%  to  more  than  half  of  the 
price  of  producing  weapons.”  (Gregory,  1989:3) 

•  Overstaffing  (Gregory,  1989:4) 

•  “Clever  but  complex  contracting  methods”  (Gregory,  1989:4) 

•  “. . .  terms  of  contracts  have  become  exceedingly  complex”  (Gregory,  1989:4) 

•  “In  trying  to  create  a  perfect  system,  refonners  took  one  from  3-5  years  in  the 
60’s  to  three  times  longer  to  do  the  same  job.”  (Gregory,  1989:8) 

•  Requirements  are  overstated  (Gregory,  1989:21) 

•  “In  the  days  before  World  War  II,  when  aircraft  companies  were  small  and  the 
costs  of  building  prototypes  were  a  few  thousand,  investors  could  take  the 
gamble...”  (Gregory,  1989:72) 
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•  Cost-plus  contracts  provide  contractors  with  little  incentive  to  minimize  costs. 

(Gregory,  1989:76) 

Gregory’s  rigorous  critique  addresses  the  military  technology  community’s 
preference  for  “clever  but  complex”  solutions,  the  lack  of  incentives  to  minimize  costs  or 
timelines,  and  the  tendency  to  oversize  everything,  from  staffing  to  management  to 
regulations  and  specifications.  In  a  nutshell,  Gregory’s  assessment  is  that  the  “mess”  is 
largely  the  result  of  a  defense  procurement  environment  that  is  too  big,  too  complicated, 
too  slow,  and  too  expensive,  and  which  seems  to  perversely  value  these  attributes.  These 
unfortunate  values  are  precisely  the  opposite  of  the  FIST  framework,  and  they  lead  to 
waste,  scandal  and  a  loss  of  credibility  for  military,  government  civilians  and  contractors 
alike. 


In  the  book’s  final  chapter,  titled  Where  Do  We  Go  From  There?,  Gregory 
suggests  the  acquisition  program  should  simplify  and  shrink,  in  order  to  reduce  timelines 
and  costs.  He  writes: 

The  acquisition  system  must  return  to  the  straightforward  principles  of  good, 
flexible,  tight  project  management  [ Simple ] — that  is,  smaller,  dedicated  teams 
[Tiny]  with  responsibility  and  authority;  expeditious  decision  making  [Fas-/];  the 
replacement  of  layers  of  review  and  oversight  [Simple]  with  short,  clear  chains  of 
command  [Tiny].  Otherwise,  little  is  in  sight  to  reduce  the  length  of  the 
acquisition  cycle — the  time  to  develop  and  field  new  weapons — the  best  and  final 
measure  of  how  well  the  process  is  working  [Fast],  (Gregory,  1989:193-194) 


Gregory  goes  on  to  advocate  shrinking  the  Office  of  the  Secretary  of  Defense, 
reducing  the  paperwork  associated  with  acquisitions  and  eliminating  layers  of 
bureaucracy  and  congressional  oversight.  These  recommendations  align  closely  with  the 
FIST  value  set,  and  are  basically  oriented  around  the  “Tiny  is  good”  value,  which  as  we 


45 


have  seen  fosters  the  adoption  of  FIST’s  other  three  components.  Sadly,  Gregory’s 
recommendations  have  not  been  implemented. 

Lessons  From  The  Development  of  Army  Systems,  2008 

Published  in  the  July  2008  issue  of  Defense  Acquisition  Review  Journal,  this 

article  examined  13  Army  systems  and  concludes  “These  data...  suggest  that  it  is  better 
to  organize  and  budget  projects  for  shorter  time  frames  whenever  possible.”  In  making 
this  conclusion,  the  authors  echo  observations  from  the  1986  Packard  Commission,  to 
name  but  one  prominent  correlating  study. 

On  their  main  finding,  the  authors  explain  “A  central  conclusion  from  this  study  is 
that  shorter  development  cycle  times  favorably  correlate  with  lower  levels  of. . .  program 
instability,  and  with  substantially  better  project  outcomes.”  (Lucas  &  Rhoades,  2008: 1 15). 
This  is  not  exactly  news  to  anyone  who  has  been  exposed  to  defense  technology 
development  projects,  but  it  is  one  more  piece  of  statistical  evidence  supporting  the 
importance  of  the  Fast  value.  It  is  indeed  good  and  important  to  have  a  short  development 
timeline. 

According  to  Lucas  and  Rhoads,  “stability  of  program  resources,  staffing,  and 
objectives  is  a  very  powerful  influence  on  the  relative  success  of  projects.”  They  go  on  to 
point  out  that  “the  longer  a  system  stayed  in  development,  the  greater  chance  it  had  to 
experience  one  or  more...  destabilizing  events.”  (Lucas  &  Rhoades,  2008:127).  Again, 
the  relationship  between  long  projects  and  increased  instability  is  logically  and  intuitively 
obvious,  but  this  article  provides  statistical  research  data  to  support  the  relationship. 

The  authors  also  assessed  the  level  of  complexity  in  each  of  the  13  systems  they 
investigated,  and  showed  a  mild  correlation  between  complexity  and  the  system’s  ability 
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to  achieve  key  outcomes.  However,  development  timelines  was  their  main  focus,  and 
they  primarily  discussed  complexity  in  terms  of  its  impact  on  schedule.  Thus,  while 
recognizing  that  “more  complex  projects  often  require  longer  development  cycles  and  are 
more  likely  to  experience  funding  difficulties,”  the  authors  conclude  their  data  shows  that 
“duration  alone  is  more  strongly  related  than  complexity  to  the  number  of  successful 
outcomes.”  (Lucas  &  Rhoades,  2008:128).  Nonetheless,  this  data  seems  to  provide  a 
certain  amount  of  support  to  the  importance  of  the  Simple  value,  as  well  as  evidence  of 
the  interrelationship  between  complexity  and  development  speed. 

It  is  also  worth  noting  that,  like  this  thesis,  the  article  focuses  on  operational 
outcomes,  not  programmatic  ones,  as  indicators  of  developmental  success.  In  fact,  the 
system’s  performance  in  the  field  is  the  only  measure  of  success  used  in  the  article.  They 
write: 

...one  of  the  strongest  relationships  between  development  duration  and  project 
outcomes  is  found  for  how  the  system  performed  in  the  field.  Six  of  the  seven 
systems  developed  in  (about)  three  years  or  less  met  or  exceeded  expectations 
when  they  were  deployed  in  DESERT  STORM,  compared  to  only  two  of  six  of 
the  longer  projects.  (Lucas  &  Rhoades,  2008:128) 

Lucas  and  Rhoades’  article  shows  that  “time  is  not  an  ally  of  systems 
development”  (Lucas  &  Rhoades,  2008:128).  In  other  words,  they  argue  that  it  is 
important  and  good  for  a  system  to  have  a  short  development  timeline,  and  for  project 
leaders  to  hold  to  the  Fast  value. 

Literature  Review  Conclusion 

There  is  a  wide  consensus  across  the  books,  articles  and  reports  reviewed  in  the 
previous  pages  -  many  military  system  development  programs  cost  too  much,  take  too 
long,  are  excessively  large  and  excessively  complex.  The  traditionally  prescribed 
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solutions  have  not  been  effective,  and  might  even  be  responsible  for  making  things  worse, 
by  requiring  complicated,  expensive,  lengthy  process  solutions  that  require  massive 
bureaucracies  to  implement  them.  A  combination  of  Theory  X  management  approaches 
and  constantly  changing  5-Year  Plans  simply  have  not  delivered  on  their  promises. 

Proposed  solutions  in  the  literature  tend  to  revolve  around  acceleration  and 
simplification  -  both  in  terms  of  the  technologies  we  acquire  as  well  as  the  bureaucracies 
that  do  the  acquisitions.  Simplicity  includes  the  reliance  on  people’s  talents  rather  than 
formal  process,  Theory  Y  management  instead  of  Theory  X,  mature  technologies  instead 
of  unproven  new  “wonder  weapons,”  quantity  instead  of  quality,  small  teams  instead  of 
large  conglomerates,  and  the  systems  engineering  principle  of  low-coupling  /  high- 
cohesion. 

These  reports  and  books  do  not  constitute  a  perfect  prescription  for  perpetual 
success.  In  particular,  Faster,  Better,  Cheaper  discusses  several  failed  missions,  pointing 
out  that  the  failure-per-attempt  rate  for  a  FBC  cohort  would  be  higher  than  under 
traditional  approaches.  The  failure -per-dollar  rate,  however,  is  much  better  under  the 
FBC  approach. 

What  this  body  of  literature  does  offer  is  a  combination  of  statistical  and 
theoretical  evidence  in  support  of  the  concept  that  the  FIST  values  often  lead  to  positive 
operational  outcomes.  It  further  indicates  that  the  most  prominent  failures  tend  to  involve 
perfectionist  project  leaders  rejecting  the  FIST  values  and  instead  viewing  complexity 
and  largeness  as  signs  of  organizational  and  technological  sophistication. 

The  literature  does  not  claim  the  FIST  values  always  will  produce  successful 
systems,  nor  that  every  failure  has  its  roots  in  the  opposite  of  the  FIST  value  set. 
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However,  the  consensus  seems  to  be  that  when  project  leaders  reject  the  FIST  value  set, 
the  operational  outcome  is  more  likely  to  be  negative  than  positive,  while  embracing  the 
FIST  values  tends  to  lead  to  reliable,  positive  operational  capabilities. 
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III.  METHODOLOGY 


This  research  seeks  to  identify  the  values  held  by  various  system  development 
organizations  and  project  teams,  for  comparison  with  the  operational  outcomes  of  the 
products  produced  by  those  teams.  In  order  to  identify  the  organization’s  values,  I  looked 
for  “value  clues”  in  a  variety  of  sources. 

Where  are  value  clues  found? 

Value  clues  are  most  easily  and  often  found  in  the  stories  told  by  the  members  of 
the  organization.  Success  stories  in  particular  are  an  excellent  way  to  identify  values, 
because  when  people  brag,  they  tend  to  brag  about  things  they  value,  things  they  think 
are  important.  As  they  tell  their  “war  stories,”  expressions  of  their  values  naturally  rise  to 
the  surface. 

The  tenn  story  here  is  used  in  a  general  sense,  referring  to  a  wide  scope  of 
narrative  expressions.  The  stories  evaluated  in  this  project  include  published  journal 
articles,  books,  case  studies,  official  reports  and  program  briefings,  academic  case  studies, 
newspaper  and  magazine  articles,  and  personal  interviews. 

Selecting  The  Stories 

Each  of  the  22  projects  evaluated  in  this  research  was  documented  in  multiple 
stories,  with  authors  ranging  from  journalists  to  project  leaders.  The  specific  stories  were 
selected  using  “theoretical  sampling  (i.e.  cases  are  chosen  for  theoretical,  not  statistical, 
reasons),”  because  in  this  method  “random  selection  is  neither  necessary,  nor  even 
preferable”  (Eisenhardt,  1989:537). 
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Eisenhardt  explains  “it  makes  sense  to  choose  cases  such  as  extreme  situations 
and  polar  types  in  which  the  process  of  interest  is  ‘transparently  observable.’  Thus,  the 
goal  of  theoretical  sampling  is  to  choose  cases  which  are  likely  to  replicate  or  extend  the 
emergent  theory...  This  sampling  plan  was  designed  to  build  theories  of  success  and 
failure.”  (Eisenhardt,  1989:537).  A  theory  of  success  and  failure  is  precisely  what  the 
FIST  research  aims  to  provide. 

Eisenhardt  and  Graebner  provide  further  support  for  theoretical  sampling  in  a 
2007  Academy  of  Management  Journal  article.  Citing  a  wide  range  of  published  articles, 
they  demonstrate  the  superiority  of  theoretical  sampling  over  random  sampling  for  case 
study  research:  “just  as  laboratory  experiments  are  not  randomly  sampled  from  a 
population  of  experiments,  but  rather,  chosen  for  the  likelihood  that  they  will  offer 
theoretical  insight,  so  too  are  cases  sampled  for  theoretical  reasons,  such  as  revelation  of 
an  unusual  phenomenon,  replication  of  findings  from  other  cases,  contrary  replication, 
elimination  of  alternative  explanations,  and  elaboration  of  the  emergent  theory.” 
(Eisnehardt  and  Graebner,  2007:25). 

The  theoretical  sampling  approach  is  further  supported  by  Schon’s  description  of 
reflective  practice.  While  the  scientific  research  method,  with  its  focus  on  theoretical 
understanding,  requires  the  use  of  statistical  sampling  methods  and  what  Schon  calls  “the 
logic  of  confirmation,”  reflective  practice,  with  its  focus  on  creating  practical  change,  is 
more  readily  served  by  using  “the  logic  of  affirmation.”  (Schon,  1983:155).  In  this 
methodology,  the  sample  need  not  be  statistically  representative.  It  simply  needs  to 
provide  sufficient  insight. 
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This  is  because  the  reflective  practice  approach  aims  to  identify  actions  (Schon 
calls  them  moves )  which  produce  satisfactory  results.  It  is  not  necessary,  or  even 
desirable,  to  ensure  the  move  is  a  unique  or  optimal  solution,  nor  that  it  is  based  on  a 
scientifically  robust  theory.  It  is  sufficient  for  the  move  to  produce  a  satisfactory  solution, 
based  on  a  practically  robust  theory. 

Schon  explains:  “Other  theories  of  action  or  models  might  also  account  for  the 
failure  of  the  earlier  move  and  the  success  of  the  later  one.  But  in  the  practice  context, 
priority  is  placed  on  the  interest  in  change  and  therefore  on  the  logic  of  affirmation.  It  is 
the  logic  of  affirmation  which  sets  the  boundaries  of  experimental  rigor.”  (Schon, 
1983:155).  The  logic  of  affirmation  basically  states  “Yes,  this  move  provided  a 
satisfactory  confirmation  of  the  hypothesis,”  but  does  not  rule  out  the  possibility  that 
other  moves,  based  on  other  hypotheses,  could  have  provided  equally  satisfactory 
outcomes  (or  better!).  This  is  contrary  to  “the  logic  of  confirmation,”  which  requires 
ruling  out  alternative  hypotheses  and  an  assurance  that  the  outcome  is  optimized  and 
unique. 

Using  the  theoretical  sampling  method,  this  thesis  examines  three  categories  of 
stories:  FIST  projects,  FIST  organizations  and  non-FIST  projects.  A  FIST  project  is  a 
single  example  of  the  FIST  values,  while  a  FIST  organization  (such  as  Lockheed’s  Skunk 
Works  division)  has  a  track  record  of  sustained  FISTiness.  The  non-FISTs  are  projects 
where  FIST  values  were  explicitly  rejected,  or  their  opposites  were  pursued.  For  each 
category,  I  analyze  projects  with  positive  and  negative  outcomes,  as  defined  in  a 
subsequent  section. 
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The  stories  focused  on  system  development  projects  (i.e.  the  F-16  Falcon),  rather 
than  technology  development  projects  (i.e.  Low  Observable  Technology).  That  is,  the 
stories  focused  on  situations  in  which  a  coherent  technology  system  was  developed  and 
delivered  for  an  operational  use,  and  did  not  examine  the  development  of  the  individual 
technological  components.  Basic  scientific  research  projects  were  also  excluded, 
although  many  of  the  NASA  stories  described  the  development  of  systems  which  were 
essentially  scientific  research  tools. 

The  FIST  Rubric 

Identifying  the  presence  of  value  clues  in  a  story  is  necessarily  somewhat 
subjective.  Some  statements  are  obvious,  such  as  “We  were  trying  to  prove  the  timeline 
could  be  radically  reduced  from  a  typical  36-month  cycle  to  7  months,”  while  other 
stories  are  more  ambiguous.  To  deal  with  this  ambiguity,  I  use  a  coding  scheme  known  as 
a  rubric  to  assess  the  presence  or  absence  of  the  FIST  values. 

A  rubric  is  a  scoring  tool  for  making  subjective  assessments.  In  instructive 
settings,  rubrics  are  often  used  by  educators  to  grade  essays.  In  the  technology 
development  world,  rubrics  are  used  in  several  maturity  models  (the  Capability  Maturity 
Model,  the  Service  Oriented  Architecture  Maturity  Model,  the  Enterprise  Architecture 
Maturity  Model,  etc)  to  detennine  an  organization’s  maturity  level.  The  FIST  rubric  can 
be  thought  of  as  measuring  a  FIST  Maturity  Level,  referred  to  as  a  FIST  Score.  The 
rubric  is  attached  in  Appendix  A  and  is  used  to  score  the  stories  (written  and  verbal). 

Each  component  of  the  FIST  model  (F,  I,  S  and  T)  is  given  a  rough  numerical 
score,  while  the  program’s  operational  outcome  is  given  a  letter  grade  (A  or  F).  The 
initial  version  of  the  rubric  used  a  spectrum  of  grades  from  A  to  F,  but  the  distinctions 
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between  A,  B  and  C  or  between  D  and  F  were  not  meaningful  in  the  context  of  this 
research  and  its  emphasis  on  “extreme  cases.”  The  simplified  binary  distinction  between 
operational  success  (A)  and  failure  (F)  was  sufficient  for  the  purpose  of  this  research. 

As  the  research  progressed,  the  rubric  was  refined  to  incorporate  new  insights  and 
observations.  Previously  scored  stories  were  then  rescored,  using  the  updated  rubric.  This 
is  consistent  with  the  iterative  tabulation  process  described  by  Eisenhardt  (1989:541-544), 
in  which  “researchers  constantly  compare  theory  and  data — iterating  toward  a  theory 
which  closely  fits  the  data,”  which  involves  “refining  the  definition  of  the  construct.” 
(Eisenhardt,  1989:541).  As  Eisenhardt  explains,  “This  verification  process  is  similar  to 
that  in  traditional  hypothesis  testing  research,”  in  which  experiments  are  repeated  when 
additional  observations  are  required.  (Eisenhardt,  1989:542) 

In  The  Academy  of  Management  Journal,  Larsson  (1993)  points  out  a  significant 
benefit  from  this  method,  namely  that  the  research  “can  be  replicated  since  both  their 
coding  schemes  [i.e.  rubric]  and  case  study  reports  [i.e.  stories]  are  available  to  other 
researchers.  Researchers  can  also  apply  the  coding  schemes  to  other  case  studies  to  cross- 
validate  or  extend  the  original  findings.”  (Larsson,  1993:1517).  This  aspect  will  be 
mentioned  again  in  the  section  recommending  further  research. 

Use  Of  Co-Evaluators 

Several  sources  (Larsson,  1993:1517;  Eisenhardt,  1989:  538)  suggest  using 
multiple  researchers  to  cross-validate  the  scores  and  provide  divergent  perspectives. 
However,  since  this  research  is  a  one-person  effort,  it  was  not  feasible  to  have  every  story 
assessed  by  multiple  readers.  Instead,  I  recruited  additional  investigators  to  apply  the 
rubric  to  a  subset  of  the  stories  consisting  of  7  systems  (approximately  1/3  of  the  overall 
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population).  The  co-investigators  are  all  current  or  fonner  AFIT  students.  They  are  listed 
below,  along  with  their  AFIT  degree  program  and  graduation  dates: 

Maj  Steven  Behm,  System  Engineering,  March  2009 

Maj  Garry  Haase,  System  Engineering,  March  2009 

Capt  Gabriel  Mounce,  Electrical  Engineering,  March  2003 

Capt  Peter  Mastro,  System  Engineering,  March  2009 

Mr.  Joseph  Simmons,  Space  Systems  Engineering,  March  2009 

This  partial  re-sampling  aims  to  enhance  confidence  in  the  findings  and  reduce 
subjective  bias,  in  accordance  with  the  methodology  described  by  Eisenhardt  (1989:538). 
The  volunteer  investigators  were  brought  in  only  after  the  rubric  was  stabilized,  and  their 
assessments  are  included  in  the  Data  section,  following  my  own  assessment. 

A  Note  On  Stories:  Accuracy  Versus  Meaning 

The  stories  evaluated  in  this  research  are  told  and  written  by  practitioners  and 
program  managers,  not  professional  historians.  Each  story  author  has  a  purpose  and  a 
point  of  view,  and  often,  the  purpose  has  nothing  to  do  with  producing  an  accurate 
historical  record.  Thus,  the  stories  analyzed  in  this  thesis  may  not  be  dependable  as 
accurate  retellings  of  historical  events.  This  is  not  problematic,  because  historical 
accuracy  in  the  stories  is  not  necessary  for  the  purpose  of  this  research. 

That  is,  a  story  need  not  be  factual  to  be  useful  as  a  source  of  value  clues.  Indeed, 
a  rigorously  researched  narrative  full  of  facts  and  figures  might  actually  be  less  useful 
than  a  more  colorful  and  less  historically  rigorous  telling.  For  the  purposes  of  this 
research,  the  most  interesting  and  relevant  aspects  might  have  nothing  to  do  with  the 
historical  facts  of  the  case,  but  rather  with  the  perceptions,  priorities,  meanings  and 
values  expressed  in  the  story. 
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As  O’Neill  explains,  “The  organizational  story  may  lack  accuracy,  but  it  does  not 
lack  meaning  to  the  recipients...”  (O’Neill,  2002:6).  O’Neill  goes  on  to  explain  that  a 
story  may  be  full  of  “embellishments,  omissions  and  differences  between  different 
versions,”  (O’Neill,  2002:  7),  but  these  do  not  diminish  the  story’s  value.  In  fact,  the 
inaccuracies  may  actually  be  significant  indicators  and  value  clues.  O’Neill  writes  “such 
stories  clearly  relate  organizational  values...  However,  these  stories  communicate  their 
messages  in  symbolic,  rather  than  literal  form.”  (O’Neill,  2002:7). 

In  other  words,  stories  have  tremendous  value  even  when  their  historical, 
empirical  accuracy  is  in  question  or  unevaluated.  Accordingly,  in  assessing  and  scoring 
the  stories,  no  attempt  was  made  to  gauge  their  historical  accuracy  with  regards  to 
various  matters  of  fact.  Even  obvious  embellishments  or  glaring  omissions  were  useful 
and  potentially  full  of  value  clues. 

Method  Validation 

Narrative-based  research  is  increasingly  popular  in  the  academic  literature, 
ranging  from  analysis  of  formal  case  studies  to  more  organic,  ethnographic  story-based 
approaches.  Not  surprisingly,  there  are  many  ways  to  investigate  and  analyze  narrative 
knowledge,  and  many  possible  coding  schemes  for  detecting  and  understanding  value 
clues.  I  chose  a  method  which  is  widely  used  by  qualitative  researchers,  particularly  in 
the  field  of  organizational  research. 

In  The  Academy  of  Management  Review,  Stanford  University’s  Kathleen 
Eisenhardt  published  a  widely-cited  article,  which  provides  “a  roadmap  for  building 
theories  from  case  study  research.”  (Eisenhardt,  1989:532).  Eisenhardt  further  explains 
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that  “case  studies  can  be  used  to  accomplish  various  aims,”  including  both  testing  and 
generating  theories  (Eisenhardt,  1989:535). 

As  previously  mentioned,  the  FIST  research  project  extends  the  definition  of 
“case  study”  to  include  a  wide  range  of  organizational  and  programmatic  narratives,  a  use 
which  is  consistent  with  Eisenhardt’s  (and  others)  use  of  the  term. 

Eisenhardt’ s  1989  article  includes  a  table  titled  Process  of  Building  Theory  from 
Case  Study  Research,  a  portion  of  which  is  reproduced  below.  These  eight  steps  (and 
accompanying  activities)  generally  describe  this  project’s  methodology: 

Table  1:  Process  of  Building  Theory  from  Case  Study  Research 

(Eisenhardt,  1989:  533) 


STEP  ACTIVITY 


1 .  Getting  Started 

Definition  of  research  question 

Possibly  a  priori  constructs 

Neither  theory  nor  hypotheses 

2.  Selecting  Cases 

Specified  population 

Theoretical,  not  random  sampling 

3.  Crafting  Instruments 
and  Protocols 

Multiple  data  collection  methods 

Qualitative  and  quantitative  data  combined 

Multiple  investigators 

4.  Entering  the  Field 

Overlap  data  collection  and  analysis,  including  field  notes 
Flexible  and  opportunistic  data  collection  methods 

5.  Analyzing  Data 

Within-case  analysis 

Cross-case  pattern  search  using  divergent  techniques 

6.  Shaping  Hypotheses 

Iterative  tabulation  of  evidence  for  each  construct 
Replication,  not  sampling,  logic  across  cases 

Search  evidence  for  “why”  behind  relationships 

7.  Enfolding  Literature 

Comparison  with  conflicting  literature 

Comparison  with  similar  literature 

8.  Reaching  Closure 

Theoretical  saturation  when  possible 

This  1989  article  is  cited  over  5,600  times,  and  Dr.  Eisenhardt  followed  it  up  with 
related  articles  in  1991  and  2007,  both  of  which  were  also  widely  cited.  The  1991  article 
expands  the  concept  of  multi-case  theory  building  versus  single-case  theory  building, 
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while  the  2007  Academy  of  Management  Journal  article,  co-written  with  Graebner,  cites 
35  other  researchers  to  provide  additional  confirmation  of  the  effectiveness  of  multiple- 
case  theory  building  methodology  and  offer  further  explanation  on  specific  methods  for 
theoretical  sampling.  Eisenhardt  and  Graebner  describe  the  method  as  “one  of  the  best  (if 
not  the  best)  of  the  bridges  from  rich  qualitative  evidence  to  mainstream  deductive 
research.”  (Eisenhardt  &  Graebner,  2007:25). 

Misconceptions  on  Case  Study  Use  and  Methodological  Bias 

Writing  in  Qualitative  Inquiry,  Flyvbjerg  (2006)  corrects  several  popular 

misunderstandings  about  case  study  research.  Two  prominent  misconceptions  are  that 
case  study  research  is  only  useful  for  generating  hypotheses,  but  not  for  testing  them,  and 
that  case  study  contains  a  bias  towards  verification.  The  following  two  conclusions  from 
his  article  are  particularly  relevant  to  this  thesis  (emphasis  added): 

1)  “The  case  study  is  useful  for  both  generating  and  testing  of  hypothesis.” 
(Flyvbjerg,  2006:  229) 

2)  “The  case  study  contains  no  greater  bias  towards  verification  of  the 
researchers’  preconceived  notions  than  other  methods  of  inquiry.  On  the 
contrary,  experience  indicates  that  the  case  study  contains  a  greater  bias 
towards  falsification  of  preconceived  notions  than  toward  verification.” 
(Flyvbjerg,  2006:237) 

On  the  question  of  bias,  Flyvbjerg  cites  “the  proximity  to  reality”  inherent  in  case 
studies — particularly  field  research  and  interview-based  studies — as  “a  prerequisite  for 
advanced  understanding.”  (Flyvbjerg,  2006:  236).  In  fact,  his  research  shows  “it  is 
falsification,  not  verification,  that  characterizes  case  study.”  (Flyvbjerg,  2006:  235). 
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Citing  extensive  previous  research  in  this  area,  he  writes,  “According  to  Campbell  (1975), 
Ragin  (1992),  Geertz  (1995),  Wieviorka  (1992),  Flyvbjerg  (1998,  2001),  and  others, 
researchers  who  have  conducted  intensive,  in-depth  case  studies  typically  report  that  their 
preconceived  views,  assumptions,  concepts,  and  hypotheses  were  wrong  and  that  the  case 
material  has  compelled  them  to  revise  their  hypotheses  on  essential  points.”  (Flyvbjerg, 
2006:235).  In  other  words,  case  study  research  is  more  likely  to  challenge  a  researcher’s 
a  priori  bias  than  to  confirm  it. 

Eisenhardt  makes  a  similar  point,  explaining  the  method  “tends  to  ‘unfreeze’ 
thinking,  and  so  the  process  has  the  potential  to  generate  theory  with  less  researcher  bias 
than  theory  built  from  incremental  studies  or  armchair,  axiomatic  deduction.  A  second 
strength  is  that  the  emergent  theory  is  likely  to  be  testable  with  constructs  that  can  be 
readily  measured  and  hypotheses  that  can  be  proven  false.”  (Eisenhardt,  1989:  546-547). 

While  this  method  has  a  documented,  demonstrable  tendency  to  falsify,  rather 
than  verify,  researcher  bias,  the  inherently  subjective  nature  of  knowledge  is  still 
unavoidable  and  worth  mentioning.  However,  subjectivity  is  not  necessarily  detrimental. 
In  fact,  Breuer  and  Roth  argue  that  “knowledge  is...  inherently  subjective,”  but  they 
“regard  the  subjective  nature  of  research  as  a  productive  opportunity”  (Breuer  and  Roth, 
2003:  5)  instead  of  a  detractive  risk.  They  go  on  to  explain  that  “all  actions  in  research 
and  interactions  during  the  research  process  can  contribute  in  a  positive  way  to  our 
understanding.”  (Breuer  and  Roth,  2003:  1 1) 

So,  we  have  one  researcher  asserting  that  case  study  research  tends  to  disprove 
biases  rather  than  confirm  them,  and  another  pointing  out  that  subjectivity  is  productive, 
not  detrimental.  Taken  together,  these  two  principles  offer  strong  support  to  the 
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qualitative,  subjective,  story-based  methodology  used  to  produce  and  test  the  FIST 
hypothesis. 
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IV.  Results  and  Analysis 


The  Systems 

This  research  encompasses  22  different  projects,  ranging  in  time  from  the  World 
War  II  era  to  the  present  day,  and  in  scope  from  NASA  planetary  exploration  to  USAF 
aircraft  to  USMC  ground  transports  to  NRO  satellite  constellations.  The  scores  and 
outcomes  are  tabulated  in  Table  2.  The  actual  data  for  each  system  is  presented  in 
Appendix  C:  Story  Data. 


Table  2:  Project  Scores  And  Grades 


System  Name 

F-Score 

1-Score 

S-Score 

T-Score 

FIST  Score  Outcome 

NASA  Pathfinder  Mission 

10 

10 

10 

10 

40 

A 

F-16  Falcon 

10 

10 

10 

10 

40 

A 

NASA  NEAR  Mission 

10 

10 

10 

10 

40 

A 

Skunkworks 

10 

10 

10 

10 

40 

A 

Pave  Low  III 

10 

10 

10 

10 

40 

A 

P-51  Mustang 

10 

5 

10 

10 

35 

A 

F-5  Freedom  Fighter 

5 

10 

10 

10 

35 

A 

A-10  Thunderbolt 

10 

10 

10 

5 

35 

A 

F-20  Tigershark 

5 

10 

10 

10 

35 

F 

AD  Skyraider 

10 

5 

10 

5 

30 

A 

XP-75  Eagle 

10 

10 

5 

0 

25 

F 

MRAP 

10 

0 

5 

0 

15 

A 

E-3  Sentry  (AW ACS) 

5 

0 

0 

0 

5 

A 

NASA  Viking  Mission 

0 

-5 

-5 

0 

-10 

A 

C-5  Galaxy 

0 

-5 

-5 

-5 

-15 

A 

Crusader  Artillery 

-5 

-5 

0 

-5 

-15 

F 

F-15  Eagle 

-5 

-5 

-5 

-5 

-20 

A 

V- 22  Osprey 

-5 

-5 

-5 

-5 

-20 

A 

F-22  Raptor 

-5 

-5 

-5 

-5 

-20 

F 

RH-66  Comanche 

-5 

-5 

-5 

-5 

-20 

F 

Future  Imagery  Architecture 

-5 

-5 

-5 

-5 

-20 

F 

Division  Air  Defense  (DIVAD) 

-5 

-5 

-5 

-5 

-20 

F 

Figure  3  charts  the  distribution  of  FIST  scores  and  grades  for  the  entire  data  set. 
Within  a  given  quadrant,  the  relative  position  within  each  grouping  is  not  indicative  of 
differences  between  systems.  That  is,  the  fact  that  the  Pathfinder  is  at  the  top  of  the 
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column  and  the  P-51  is  at  the  bottom  is  an  artifact  of  the  listing,  and  does  not  imply  a 
higher  or  lower  score. 


-20  -5  10  25  40 


SCORE 


Figure  3:  Project  Score  and  Grade  Distribution 


As  discussed  in  the  methodology  section,  projects  were  selected  using  a 
theoretical  sampling  approach.  Unlike  statistical  sampling  methods,  theoretical  sampling 
does  not  involve  collecting  a  representative  sample  across  the  entire  field.  The  objective 
of  theoretical  sampling  is  not  to  quantify  the  density  of  project  types  and  outcomes  in  a 
given  population.  Instead,  theoretical  sampling  allows  a  researcher  to  identify  boundary 
cases,  to  illustrate,  highlight  and  understand  various  aspects  of  the  hypothesis,  theory  or 
subject  under  examination. 
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Accordingly,  the  density  of  projects  in  various  regions  of  Figure  3  is  not 
indicative  of  the  statistical  distribution  of  DoD  or  NASA  technology  development 
projects.  The  relative  lack  of  mid-range  scores  and  grades  is  similarly  an  artifact  of  this 
research’s  deliberate  predisposition  towards  extreme  projects,  and  should  not  be  taken  as 
evidence  that  such  mid-range  outcomes  do  not  exist.  The  selection  of  extreme  projects 
was  deliberate  and  consistent  with  the  methodology  guidelines.  As  Laufer  explains, 
“Excellence  [and  failure]  is  a  better  teacher  than  mediocrity.”  (Laufer,  2009). 

Figure  3  indicates  the  existence  of  both  high-score  failures  and  high-score 
successes  in  the  data  sample,  as  well  as  low-score  failures  and  low-score  successes.  That 
is,  all  four  comers  of  the  chart  are  populated  with  at  least  some  systems,  which  indicates 
the  FIST  approach  is  neither  a  guarantee  of  success,  nor  does  it  have  a  monopoly  on 
success.  This  also  shows  that  the  non-FIST  approach  neither  dooms  a  project  to  failure 
nor  guarantees  a  happy  ending. 


High  Score  Successes 

Projects  with  a  score  of  25  or  higher  on  the  FIST  rubric  are  designated  High  Score 
projects.  The  9  successful  projects  listed  below  met  these  criteria: 

1 .  NASA’s  Pathfinder  Mission 

2.  F- 16  Falcon 

3.  NASA’s  NEAR  Mission 

4.  Skunkworks 

5.  Pave  Low  III 

6.  P-51  Mustang 

7.  F-5  Freedom  Fighter 

8.  A- 10  Thunderbolt  II 

9.  AD  Skyraider 

NASA’s  Pathfinder  system  is  an  excellent  example  of  FIST-driven  success. 
Project  leaders  explicitly  affirmed  the  entire  FIST  value  set,  and  there  was  wide 
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consensus  that  their  emphasis  on  being  fast,  inexpensive,  simple  and  tiny  directly 
contributed  to  the  project’s  overwhelming  success. 

Lockheed’s  legendary  Skunkworks  unit  is  the  prototypical  High-Score 
organization.  It  has  a  track  record  of  delivering  high-perfonnance,  high-impact  aircraft  on 
remarkably  short  timelines  and  small  budgets.  They  use  very  simple  tools  and  approaches, 
and  a  small  team  of  talented,  dedicated  personnel,  whom  they  shelter  from  the  traditional 
bureaucratic  requirements  and  procedures  as  much  as  possible. 

High  Score  Failures 

High  Score  failures  are  those  projects  which,  while  achieving  a  high  FIST  score, 
earned  an  F  on  the  outcome  portion  of  the  FIST  rubric,  indicating  a  failure  to  deliver  a 
system  which  met  the  users  needs  or  which  grossly  exceeded  the  cost  and  schedule 
estimates.  Two  project  met  these  criteria:  Northrop’s  F-20  Tigershark  and  the  WWII  P-75 
Eagle.. 

The  F-20  Tigershark  is  an  excellent  example  of  a  FIST-driven,  High  Score  failure. 
Not  only  was  the  F-20  a  failure,  the  FIST  values  are  very  possibly  the  primary  cause  of 
the  failure.  That  is,  it  appears  the  explicit  presence  of  the  FIST  values  was  one  of  the 
main  reasons  for  Northrop’s  failure  to  sell  a  single  F-20’s. 

The  Tigershark  was  designed  to  be  an  inexpensive,  simple  export  fighter,  and 
Northrop  succeeded  in  achieving  that  objective.  It  was  marketed  as  a  simple  to  use,  low- 
cost  alternative  to  advanced  jet  fighters,  suitable  for  countries  which  could  not  handle  the 
costs  and  complexities  of  a  system  like  the  F-16  (which  was  originally  not  available  for 
sale  to  foreign  nations  anyway).  By  taking  this  approach,  Northrop  inadvertently  planted 
the  seeds  of  the  F-20’s  destruction. 
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Northrop’s  approach  might  have  been  effective  under  the  Carter  Administration’s 
restrictions  on  foreign  weapon  sales,  but  when  the  Reagan  Administration  approved 
foreign  sales  of  the  F-16,  the  F-20  was  widely  viewed  as  a  poor  substitute.  While  the 
Tigershark  would  likely  have  been  entirely  adequate  for  the  defense  needs  of  many 
smaller  countries,  the  opportunity  to  own  a  fleet  of  F-16’s,  however  unnecessary,  was 
simply  too  tempting.  Buying  F-16’s  was  a  way  for  countries  to  demonstrate  that  they 
were  serious  powers  and  capable  of  defending  themselves  with  front-line  fighter  jets, 
whether  they  could  actually  afford  such  purchases  or  not,  and  regardless  of  whether  the 
F-20  could  have  satisfied  their  actual  defense  needs.  Potential  customers  derogatorily 
viewed  the  F-20  as  a  “lightweight,”  (not  to  be  confused  with  the  complimentary 
designation  of  the  F-16  as  a  “lightweight  fighter”  early  in  its  development). 

This  highlights  one  of  the  dangers  of  using  the  FIST  value  set.  The  FIST  values 
may  very  well  be  effective  guidelines  for  efficiently  developing  effective,  reliable 
technology  systems.  However,  marketing  these  High  Score  systems  requires 
synchronization  with  the  customer’s  values.  If  a  customer  views  complexity  and  a  high 
price  tag  as  signs  of  sophistication,  then  a  High-Score  system  (such  as  the  F-20)  may 
appear  to  be  a  mere  toy  or  an  insufficiently  effective  alternative. 

It  should  be  noted  that  the  F-20’s  testing  was  limited  and  its  performance  was 
never  rigorously  tested  or  proven.  The  world  will  never  know  how  good  or  bad  an 
airframe  it  was.  However,  the  limited,  initial  evaluations  all  indicated  the  Tigershark  was 
reliable,  effective  and  performed  well. 
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Low  Score  Projects 

Projects  with  a  score  of  -5  or  lower  on  the  FIST  rubric  are  designated  Low  Score 


projects.  Low  Score  projects  typically  exhibited  a  strong  commitment  to  high-level 
performance  and  the  inclusion  of  high  technology  instead  of  the  FIST  values.  Project 
leaders  in  this  category  were  generally  willing  to  sacrifice  cost,  schedule,  size  and 
simplicity  in  the  pursuit  of  cutting-edge  capabilities. 

The  9  projects  listed  below  met  these  criteria: 

1.  Viking 

2.  C-5  Galaxy 

3.  F-15  Eagle 

4.  V-22  Osprey 

5.  F-22  Raptor 

6.  Crusader  Artillery 

7.  RH-66  Comanche 

8.  Future  Imagery  Architecture  (FIA) 

9.  Division  Air  Defense  (DIVAD  -  aka  Sgt.  York) 

The  Low-Score  projects  ranged  from  the  highly  successful  C-5  A,  F-15  and 
NASA’s  Viking  mission  to  Mars  to  the  spectacular  failures  of  the  Comanche,  Crusader 
and  DIVAD. 


Low  Score  Successes 

Low  Score  systems  like  the  F-15  and  C-5A  successfully  provided  effective 
operational  capabilities,  but  typically  long  after  the  originally  anticipated  delivery  date, 
and  with  a  price  tag  much  higher  than  the  initial  budget.  Project  leaders  typically 
emphasized  the  importance  of  delivering  a  cutting-edge  technology,  which  tended  to 
introduce  both  delays  and  cost  increases.  So,  while  the  low-score  projects  are  clearly 
successful  from  an  operational  perspective,  their  failure  to  deliver  on  time  and  on  budget 
diminishes  their  value  as  an  exemplar  for  project  leaders  to  imitate. 
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NASA’s  Viking  mission  to  Mars  was  a  resounding  success,  but  its  costs 
effectively  prevented  NASA  from  attempting  to  repeat  the  mission  for  two  decades.  It 
also  reduced  the  amount  of  resources  available  for  other  projects.  Thus,  while  it  was  a 
tactical  success  for  the  Viking  team,  when  viewed  from  the  strategic  perspective  of 
NASA  as  a  whole,  its  contribution  may  be  described  in  significantly  less  positive  terms. 

Low  Score  Failures 

When  low  score  projects  fail,  the  cause  can  generally  be  found  in  the  counter¬ 
productive  values  expressed  by  the  project  leaders.  Whether  it  was  a  preference  for  the 
superficial  appearance  of  programmatic  speed,  a  detennination  to  incorporate  bleeding- 
edge  technology,  or  an  overdeveloped  fondness  for  complexity,  these  values  tended  to 
undermine  the  goal  of  delivering  a  system  that  is  “available  when  needed  and  effective 
when  used.”  Often,  project  leaders  take  so  long  to  develop  the  system  that  it  is  obsolete 
before  it  is  delivered,  as  with  the  Crusader  artillery.  Other  times  the  budget  swells  to 
insupportable  levels,  which  happened  to  the  Comanche  helicopter,  or  the  system  gets  too 
complex  to  be  usable,  as  happened  with  the  DIVAD  and  FIA  projects. 

The  interviews 

The  research  plan  for  this  project  originally  included  a  number  of  interviews  with 
experienced  program  managers  and  project  leaders.  However,  after  accomplishing  the 
first  few  interviews,  it  was  clear  the  benefits  of  such  an  approach  were  limited  and  plans 
for  further  interviews  were  abandoned. 

Although  the  interviews  were  enlightening  and  enjoyable,  they  did  not  produce 
the  kind  of  data  that  could  be  readily  incorporated  into  this  research.  In  the  interviews,  I 
was  unable  to  glean  a  level  of  granularity  and  insight  equivalent  to  what  I  found  in  the 
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written  stories.  The  fault  had  nothing  to  do  with  the  subjects  and  everything  to  do  with 
the  interviewer.  A  more  experienced  or  savvy  interviewer  would  certainly  be  able  to 
produce  a  more  insightful  set  of  questions  and  answers. 

One  interview  was  a  notable  exception.  A  co-evaluator  and  I  sat  down  with  BGen 
Craig  Olson,  the  Vice  Commander  of  Aeronautical  Systems  Center.  BGen  Olson  was 
fonnerly  the  Deputy  Program  Manager  and  Materiel  Wing  Director  for  the  V-22  Joint 
Program  Office,  and  our  interview  centered  on  his  time  working  on  the  V-22.  Notes  from 
this  interview  were  incorporated  into  the  V-22  assessment  and  shed  much  light  on  the 
Osprey’s  operational  effectiveness. 
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V.  Discussion 


While  the  FIST  value  set  does  not  guarantee  a  positive  outcome  on  every  system 
development  project,  the  data  indicates  the  FIST  approach  conveys  several  significant 
benefits.  Specifically,  the  FIST  values  enhance  project  stability,  increase  the  project 
leader’s  control  and  accountability,  optimize  failure,  foster  “luck,”  and  facilitate  learning. 
Ultimately,  the  FIST  values  seem  to  support  the  goal  that  a  system  be  available  when 
needed  and  effective  when  used.  Unlike  many  common  measures  of  programmatic 
performance,  this  two-part  goal  is  shared  by  both  project  developers  and  end  users  alike. 

FIST  Enhances  Project  Stability 

Project  leaders  wage  a  constant  battle  against  instability  and  uncertainty,  whether 
caused  by  changes  in  technology,  changes  in  the  environment  or  changes  in  the 
political/financial  situation.  These  changes  are  often  unforeseen  and  unforseeable,  and  as 
Taleb  points  out,  “the  unexpected  almost  always  pushes  in  a  single  direction:  higher  costs 
and  a  longer  time  to  completion.”  (Taleb,  2007:157).  By  pursuing  the  Fast,  Inexpensive, 
Simple  and  Tiny  values,  FIST  provides  a  way  for  project  leaders  to  inject  stability  across 
several  fronts. 

Fast  Stability 

Program  instability  comes  from  many  different  sources,  but  the  primary  origin  for 
all  these  sources  is  simply  time.  Given  enough  time,  new  discoveries  and  breakthroughs 
will  render  previous  technologies  obsolete.  Old  enemies  are  defeated  while  new  enemies 
emerge.  Political  leaders  come  and  go,  as  do  program  managers  and  project  leaders. 
Economies  expand  and  contract.  Each  of  these  changes  can  cause  changes  to  a  project’s 
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structure,  objective,  funding,  design,  priority  and  schedule.  Our  inability  to  accurately 
predict  these  changes  contributes  to  the  cost  and  schedule  overruns  which  are  so 
prevalent  in  DoD  and  NASA  projects.  Further  complicating  matters  is  the  high  “impact 
of  forecast  degradation  over  long  time  periods.”  (Taleb,  2007:157).  Left  uncorrected,  a 
small  forecasting  error’s  impact  grows  over  time,  as  shown  in  Figure  4  below. 


Figure  4:  Impact  of  Error  over  Time 


This  notional  diagram  shows  that  the  near-tenn  delta  between  actual  and 
projected  is  much  smaller  than  the  far-term  delta.  The  factor  under  consideration  could  be 
cost,  schedule,  risk  or  any  number  of  other  forecastable  items. 

The  impact  of  long-term  forecast  degradation  was  clearly  seen  in  the  F-22  project. 
During  the  twenty-six  years  it  spent  in  development,  countless  technologies  emerged  and 
were  replaced,  requiring  significant,  costly  and  time-consuming  modifications  to  the 
requirements  and  design.  More  significantly,  between  the  project’s  inception  in  1981  and 
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its  IOC  in  2005,  the  Soviet  Union  collapsed  and  A1  Quaida  emerged  as  America’s  prime 
threat,  dramatically  changing  the  shape  of  the  threat  environment  from  what  was 
envisioned  at  the  Raptor’s  inception. 

These  changes  (and  many  others)  increased  the  F-22’s  cost,  delayed  its  delivery, 
shrunk  the  size  of  the  fleet  and  stunted  its  operational  effectiveness  to  the  point  that  for 
the  Raptor  has  not  flown  a  single  combat  mission,  despite  the  US  being  actively  engaged 
in  conflicts  in  both  Iraq  and  Afghanistan  for  the  three  years  since  it  went  IOC.  (Charette, 
2008:37) 

In  contrast  to  traditional  development  approaches  such  as  those  used  on  the  F-22 
program,  the  FIST  approach  reduces  a  project’s  exposure  to  the  unexpected  and 
minimizes  this  particular  source  of  instability.  Project  leaders  who  embrace  the  FIST 
values  insist  on  short  timelines  and  make  decisions  that  support  rapid  delivery  of  the 
required  capabilities.  By  valuing  speed,  they  aim  to  deliver  capabilities  before  too  many 
changes  are  manifest.  This  does  not  mean  all  change  will  be  avoided,  but  both  the 
quantity  and  significance  of  the  change  will  be  greatly  diminished. 

Inexpensive  Stability 

While  a  short  timeline  provides  a  great  deal  of  a  FIST  project’s  stability,  a  small 
budget  also  conveys  stability.  This  is  largely  because  FISTy  projects  are  not  very 
tempting  sources  of  funds  when  budget  cuts  come  down  the  line,  for  the  simple  reason 
that  a  FIST  project’s  budget  is  already  small.  It  has  essentially  been  pre-cut,  and  further 
decreases  to  a  Tiny  budget  are  likely  to  be  seen  as  not  only  unfair  but  also  unproductive 
and  unlikely  to  make  a  dent  in  the  organization’s  overall  finances.  It  simply  makes  more 
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sense  for  budget  cutters  to  go  after  deeper  pockets  and  projects  that  are  better  able  to 
survive  a  reduction. 

The  combination  of  a  short  timeline  and  small  budget  means  the  project  is  more 
likely  to  be  fully  funded  from  the  start,  rather  than  having  to  deal  with  the  uncertainty  of 
requiring  budget  authorities  to  authorize  future  (large)  budgets.  The  near-term  delivery 
schedule  also  means  budget  cutters  are  less  likely  to  cut  a  current-year  budget  and 
promise  to  “repay”  it  with  future  dollars,  because  the  project  is  potentially  slated  to 
deliver  before  the  funds  can  be  repaid. 

Finally,  new  regulations  tend  to  focus  on  high-profde,  big-ticket  projects.  FISTy 
projects  with  sufficiently  small  budgets  can  remain  appropriately  below  the  radar  and 
thus  minimize  exposure  to  the  unexpected  changes  that  often  accompany  new  regulations. 

Simple  Stability 

Simplicity  also  fosters  stability,  both  technically  and  organizationally.  From  a 
technical  perspective,  the  Simple  value’s  emphasis  on  mature  technology  tends  to  reduce 
the  uncertainty  and  instability  inherent  in  cutting-edge,  not-quite-proven  technologies. 
The  principle  of  low  coupling  /  high  cohesion,  which  is  also  a  key  technical  element  of 
simplicity,  means  that  the  impact  of  change  is  localized,  with  limited  ripple  effect 
through  the  rest  of  the  system.  Further,  a  simpler,  more  streamlined  organization  tends  to 
have  faster  and  clearer  communication  than  a  large,  complex  organization.  This 
communication  clarity  reduces  the  amount  of  instability  caused  by  miscommunication 
and  bureaucratic  inefficiency. 

The  bottom  line  is  that  the  FIST  values  help  projects  present  a  smaller  target  to 
the  forces  of  change,  whether  those  forces  are  financial,  technical,  political,  legislative  or 
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some  other  category.  This  does  not  mean  FIST  makes  projects  more  resistant  to  change  - 
rather,  they  are  simply  confronted  with  fewer  instances  where  change  is  necessary. 

FIST  Increases  Project  Leader’s  Control  And  Accountability 

Technology  systems  development  efforts  typically  include  a  wide  range  of 

stakeholders.  For  the  DoD  and  NASA,  these  stakeholders  include  many  layers  of 
bureaucracy  and  several  branches  of  government.  Involvement  by  such  an  extensive 
community  often  limits  the  amount  of  influence  a  project  leader  can  have  on  the  project 
he  or  she  is  responsible  for.  This  is  unfortunate  because  the  project  leader  arguably  has 
the  most  accurate,  relevant  and  timely  infonnation,  particularly  when  compared  with 
individuals  who  are  several  layers  removed  from  the  project  or  who  are  from  other 
organizations  entirely.  By  enhancing  the  project  leader’s  control,  FIST  also  increases  the 
project  leader’s  accountability  for  the  outcome. 

Writing  in  Defense  Acquisition  Review  Quarterly,  Christensen,  Searle  and 
Vickery  examine  the  question  of  programmatic  influence,  drawing  an  interesting 
distinction  between  cost  growth  and  cost  overruns  on  acquisition  programs.  They  define 
cost  growth  as  “the  difference  between  the  initial  budget  and  the  final  cost  of  the 
program,”  while  cost  overruns  are  defined  as  “the  difference  between  a  contract’s  final 
budget  and  final  cost.”  (Christensen  et  al,  1999:253-254;  emphasis  added). 

The  authors  observe  that  the  traditional  definition  of  cost  growth  fails  to  make  a 
distinction  between  factors  which  a  program  manager  can  influence  and  those  factors 
which  are  beyond  a  project  leader’s  scope  of  influence  (i.e.  changes  in  technology, 
changes  in  the  threat  environment,  congressional  decisions,  etc).  They  suggest  project 
leaders  have  little  influence  on  cost  growth,  and  thus  should  not  be  held  accountable  for  it. 
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Cost  overruns,  on  the  other  hand,  derive  from  factors  which  project  leaders  can  influence. 
So,  they  recommend  cost  overruns  as  a  more  appropriate  metric  for  determining 
programmatic  efficiency. 

It  is  indeed  important  to  distinguish  between  factors  a  project  leader  can  influence 
and  those  that  are  outside  their  control,  so  there  is  much  merit  to  this  argument.  However, 
project  leaders  actually  have  more  influence  over  the  so-called  external  factors  than 
Christensen  et  al  seem  to  think.  It  is  not  necessary  to  simply  accept  cost  growth  in  an 
acquisition  project  as  an  unavoidable  fact  of  life,  unless  we  also  accept  decades-long 
timelines  as  inevitable  and  unavoidable.  As  the  previous  section  explained,  by  limiting 
the  amount  of  time  spent  on  a  project,  FIST  gives  project  leaders  the  opportunity  to 
minimize  many  of  the  destabilizing  external  forces  that  lead  to  cost  growth. 

The  introduction  of  change  from  the  external  environment  is  beyond  the  control 
of  the  project  leader.  However,  by  using  a  small  budget,  a  short  schedule  and  a 
streamlined  team,  FIST  minimizes  the  program’s  exposure  to  these  changes,  retaining 
control,  influence  and  accountability  at  the  local  project  leader’s  level  instead  of  allowing 
external  actors  to  take  control  of  the  project.  In  fact,  the  project  leader’s  influence  is  often 
inversely  proportional  to  the  size  of  the  budget  and  schedule.  Nowhere  is  this  seen  more 
starkly  than  when  a  project  needs  to  be  cancelled. 

FIST  Projects  Are  Easier  To  Cancel 

The  V-22  Osprey  story  illustrates  an  interesting  benefit  of  FIST  projects. 
Specifically,  failing  projects  are  easier  to  kill  if  they  are  FISTy  instead  of  big,  complex, 
and  expensive. 
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Secretary  of  Defense  Dick  Cheney  learned  firsthand  exactly  how  difficult  it  is  to 
cancel  a  big  project  that’s  gone  bad.  As  Time  Magazine  explained,  the  Osprey 
proved  impossible  to  kill,  thanks  to  lawmakers  who  rescued  it  from  [SECDEF]  Cheney’s 
ax  time  and  again  because  of  the  home-district  money  that  came  with  it.”  (Thompson, 
2007:1).  The  Center  for  Strategic  &  International  Studies  put  it  more  bluntly,  observing 
“the  V-22  program  has  nearly  2,000  suppliers  in  over  40  states,  and  created  jobs  in  276 
congressional  districts.”  (Cordesman  and  Kaeser,  2008:26)  It  is  not  surprising,  therefore, 
that  the  Osprey  survived  multiple  attempts  by  the  Secretary  of  Defense  to  cancel  it. 

The  simple  reality  is  that  big,  complex  and  expensive  projects  like  the  V-22  tend 
to  create  a  lot  of  jobs,  in  multiple  congressional  districts.  This  is  considered  good  for 
employment,  but  it  means  the  DoD  is  sometimes  forced  to  continue  projects  it  would 
prefer  to  cancel.  Elevating  this  kind  of  decision  making  ability  to  the  Congressional  level 
effectively  reduces  the  project  leader’s  influence  and  accountability.  The  DoD  could 
avoid  this  problem  by  not  permitting  these  conditions  in  the  first  place. 

Congressmen  and  Senators  are  appropriately  concerned  with  protecting  jobs  in 
their  districts  and  representing  the  interests  of  their  constituencies.  This  is  what  they  are 
elected  to  do,  and  such  protective  actions  are  entirely  consistent  with  their  legitimate 
responsibilities.  However,  when  the  DoD  launches  a  big  project,  large  numbers  of  jobs 
are  tied  to  a  single  system.  Cancelling  such  a  system  causes  pain  in  many  locations,  and 
Congress  naturally  tries  to  prevent  the  negative  economic  impact.  At  that  point,  the  DoD, 
even  at  the  SECDEF  level,  often  finds  it  difficult  or  even  impossible  to  cancel  the  project, 
even  if  it  is  having  as  many  problems  as  the  V-22  was. 
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In  contrast,  FIST  projects  are  deliberately  too  small  to  spread  across  a  wide  range 
of  Congressional  districts.  By  design,  FIST  projects  are  developed  by  Tiny,  co-located 
organizations  (which  may  be  part  of  a  larger  entity,  as  with  Lockheed  Martin’s 
Skunkworks  division).  When  a  FIST  project  begins  to  go  south,  it  can  be  cancelled 
without  any  given  district  losing  very  many  jobs,  relatively  speaking,  and  before  much 
time  has  passed.  Similarly,  the  amount  of  money  being  spent  on  any  given  FIST  project 
is  relatively  low  and  is  therefore  easier  to  write-off  as  a  research  investment,  leading  to 
less  hand-wringing  over  waste  than  the  cancellation  of  a  $10  billion  project. 

Congress  is  not  the  problem  -  they  are  simply  doing  what  they  are  elected  to  do. 
The  problem  is  the  DoD’s  willingness  to  pursue  big,  expensive,  complex  and  lengthy 
projects.  Indeed,  a  system  does  not  end  up  with  2,000  suppliers  in  40  states  by  accident. 
That  is  very  likely  a  deliberate,  cynical  attempt  to  manipulate  the  system,  taking 
advantage  of  Congress’s  legitimate  priorities  and  relying  on  political  pressure  to  keep  a 
project  alive,  rather  than  the  project’s  technical  and  operational  merits. 

Projects  of  all  types  go  bad,  and  when  they  do,  cancelling  them  should  be  an 
option.  It  seems  logical  that  the  Secretary  of  Defense  should  be  able  to  decide  whether 
the  military  needs  a  particular  system,  without  being  trumped  by  Congress.  Spreading  a 
project  across  276  districts  makes  the  system  “too  popular  to  be  killed”  and  effectively 
takes  the  cancellation  option  off  the  table.  This  leads  to  excessive  spending  on  projects 
the  DoD  does  not  want,  reduces  the  project  leader’s  influence  and  accountability,  and 
damages  the  credibility  of  the  defense  acquisition  community. 
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FIST  Optimizes  Failure 

Neither  the  traditional  approach  to  system  development  nor  the  FIST  approach 
guarantees  a  successful  outcome.  Given  enough  attempts,  a  certain  amount  of  failure  is 
inevitable,  regardless  of  approach.  But  while  we  cannot  avoid  failure  entirely,  we  can 
influence  the  kinds  of  failure  we  experience.  Nicholas  Taleb  suggests  striving  to  create 
“situations  where  favorable  consequences  are  much  larger  than  unfavorable  ones,”  (Taleb, 
2007:210).  That  is,  we  ought  to  pursue  situations  where  the  benefits  of  a  positive 
outcome  significantly  outweigh  the  cost  of  a  negative  outcome.  One  of  the  ways  to  do 
this  is  to  minimize  exposure  to  loss.  The  other  is  to  ensure  that  any  negative  outcomes 
become  learning  experiences  and  building  blocks  for  future  endeavors. 

The  ideal  failure,  therefore,  is  one  in  which  little  is  lost  and  much  is  learned.  Such 
a  failure  could  be  termed  an  optimal  failure.  A  negative  failure,  in  contrast,  is  one  in 
which  much  is  lost  and  little  is  learned.  Table  3  illustrates  the  differences  between 
optimal  failure  and  negative  failure. 


Table  3:  Failure  Types 


Exposure  to  Loss 

Opportunity  to  Learn 

Optimal 

Low 

High 

Negative 

High 

Low 

FIST  failures  tend  to  be  optimal  failures.  They  are  discovered  before  much  time 
and  money  is  expended,  so  losses  are  small.  They  also  have  a  high  probability  of 
conveying  meaningful  lessons,  because  on  a  short  schedule,  project  leaders  are  more 
easily  able  to  witness  the  impacts  of  their  decisions  and  leam  from  their  experience. 
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Unfortunately,  negative  failures  are  arguably  the  only  kind  of  failure  a  major 
defense  acquisition  program  (MDAP)  can  experience,  given  the  typical  MDAP’s 
enonnous  budget  and  decades-long  schedule.  Every  time  an  MDAP  fails,  it  fails 
spectacularly,  costing  billions  and  teaching  too  little,  too  late. 

Projects  below  the  MDAP  threshold  can  also  experience  negative  failures,  given 
enough  years  and  dollars.  Whenever  large  quantities  of  time  and  money  are  expended 
before  the  outcome  is  observable,  the  project  is  exposed  to  significant  loss.  If  a  large 
percentage  of  participants  have  moved  on  to  other  projects  and/or  retired  before  the 
failure  is  observed,  the  opportunity  to  leam  is  low.  Even  if  the  original  decision  makers 
are  still  around  and  directly  witness  the  consequences  of  their  actions,  it  is  often  too  late 
to  apply  the  lessons  very  much  further.  This  is  because  learning  requires  both  observation 
of  the  phenomena  and  timely  reflection  followed  by  action,  neither  of  which  are  likely  in 
big,  lengthy,  expensive  projects. 

From  1983  to  2004,  the  US  Anny  spent  $7B  developing  the  Comanche  helicopter, 
then  cancelled  the  program  and  had  zero  aircraft  to  show  for  their  troubles.  Commenting 
on  the  cancellation  in  Feb  2004,  LtGen  Richard  Cody,  deputy  chief  of  staff,  G-3,  said  “If 
you  told  me  six  months  ago  that  I  would  be  standing  here  saying  the  Anny  no  longer 
needs  the  Comanche  helicopter,  I  wouldn’t  have  believed  you.”  (Merle,  2004)  That 
admirably  honest  statement  highlights  the  inherent  difficulty  in  learning  from  experience 
on  a  long  project,  and  shows  that  we  really  don’t  know  what  the  lessons  will  be  until  the 
story  is  finished. 

For  nearly  2 1  years,  the  Army  apparently  thought  things  with  the  Comanche  were 
satisfactory,  maybe  even  worthy  of  imitation.  They  didn’t  have  the  opportunity  to  leam 
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the  true  lessons  of  the  Comanche  until  it  was  cancelled.  Until  that  moment  in  2004, 
there’s  a  good  chance  the  Army  was  learning  -  and  teaching  -  the  wrong  lessons. 

The  FIST  approach  requires  a  significantly  different  perspective  on  failure  than 
the  standard  methodology  used  on  systems  like  the  Comanche.  In  the  traditional  approach, 
it  makes  sense  to  measure  failure  rates  on  a  per-attempt  basis  (i.e.  failures  per-cohort  or 
per-portfolio),  because  each  attempt  is  expensive  and  takes  a  long  time  to  realize.  But 
when  attempts  are  quick  and  inexpensive,  as  in  the  FIST  approach,  a  relatively  high 
failure  rate  is  more  acceptable,  or  perhaps  even  irrelevant.  Indeed,  a  relatively  high  per- 
attempt  failure  rate  should  perhaps  even  be  demanded.  Interestingly,  when  the  FBC 
initiative  began,  NASA  Administrator  Dan  Goldin  warned  against  excessively  high 
success  rates.  He  told  the  Jet  Propulsion  Laboratory’s  staff  “[A]  project  that's  20  for  20 
isn’t  successful.  It's  proof  that  we’re  playing  it  too  safe."  (JPL  History  -  The  90’s).  Thus, 
the  success  rate  of  the  FIST  approach  is  best  measured  on  a  per-dollar  basis,  rather  than 
per-attempt,  with  some  accounting  made  for  the  benefits  of  learning  that  optimized 
failures  convey. 

FIST  Enhances  Learning 

Failure  is  not  the  only  way  to  leam,  although  it  is  one  of  the  most  fruitful.  A 
diverse  set  of  experiences  can  also  help  produce  agile-minded,  well-rounded  and  creative 
project  leaders.  Unfortunately,  the  traditional  development  approach,  particularly  as 
currently  practiced  in  the  field  of  manned  aircraft  development,  is  trending  in  the 
opposite  direction. 

Figure  5  is  from  a  2005  presentation  by  senior  Lockheed  Martin  personnel  to  the 
Defense  Acquisition  Performance  Assessment  Panel  (Kubasic  &  Heath,  2005:4).  Using 
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data  from  a  1992  RAND  Corp  study  (Drezner,  et  al,  1992),  it  shows  that  in  today’s 
environment,  a  typical  40-year  career  span  encompasses  far  fewer  military  aircraft 
programs  than  in  generations  past.  The  17  years  since  that  report  was  published  have 
confirmed  its  predictions. 

In  a  similar  trend,  the  number  of  aerospace  companies  available  to  choose  from 
has  decreased  significantly.  For  example,  in  1970,  when  the  A- 10  was  being  developed, 
“twelve  companies  were  selected  to  receive  the  RFP...  six  companies  responded  with 


proposals.”  (Jacques  &  Strouble,  2008:32)  Today,  the  Air  Force  is  basically  limited  to 


three:  Lockheed-Martin,  Boeing  and  Northrop,  and  when  it  comes  to  building  manned  jet 


lighters,  these  companies  are  more  likely  to  cooperate  than  compete,  which  is  a  mixed 


blessing  at  best. 


(Vertical  Bars:  Military  Aircraft  Program  Starts,  Horizontal  Bars:  Typical  40  Year  Career  Span) 
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Figure  5:  Declining  Experience  Levels 
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This  decrease  in  the  number  of  aircraft  being  developed  (and  in  the  number  of 
aircraft  developers)  has  a  negative  effect  on  the  industry’s  overall  experience  level.  As 
the  original  RAND  study  points  out: 

An  engineer  who  began  his  career  in  the  1950s  would  have  worked  in  an  industry 
that  developed  and  flew  84  new  designs  before  he  retired.  An  engineer  who 
started  in  the  1960s  will  see  only  40  new  designs  fly.  The  situation  gets 
significantly  worse  with  time.  The  entry-level  designers  and  engineers  in  the 
1980s  and  1990s  will  be  the  senior  technical  and  management  staff  in  the  2010s, 
and  their  scope  and  breadth  of  design  experience  will  fall  significantly  short  of  the 
experience  of  today’s  senior  technical  staff.  Furthennore,  the  future  work 
environment  for  these  engineers  will  lack  the  opportunity  for  cross-fertilization  of 
ideas  and  knowledge  between  firms  and  projects.  (Drezner,  et  al,  1992:48,50) 

Unlike  the  community  in  the  1950’s  and  1960’s,  today’s  aircraft  engineers, 

program  managers  and  other  project  leaders  have  far  fewer  opportunities  to  gain 

experience  and  perspective,  to  experiment  and  learn,  or  even  to  talk  with  other  people 

whose  experiences  in  manned  aircraft  design  diverge  from  their  own.  This  situation  is  not 

limited  to  aircraft,  as  many  defense  related  fields  are  tending  towards  producing  small 

numbers  of  big  projects  rather  than  large  numbers  of  smaller  projects.  A  similar  chart 

could  easily  be  made  for  tanks,  artillery  or  naval  ships.  This  reduced  exposure  to  different 

projects  provides  limited  opportunities  to  leam,  which  has  a  negative  impact  on  the 

community’s  overall  perfonnance. 


However,  the  situation  with  UAV’s  is  far  different,  as  Figure  6  shows.  In  fact,  the 
UAV  development  environment  appears  to  be  the  inverse  of  the  manned  aircraft  situation. 
The  result  is  a  wide  range  of  opportunities  for  experimentation  and  learning  among 
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developers,  while  simultaneously  providing  innovative,  world-class  capabilities  for  the 
US  military  across  a  broad  spectrum  of  operations. 


UAV  Evolution  -  Where  are  we? 
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Figure  6:  UAV  Evolution 


The  RAND  study’s  concerns  about  the  decrease  in  new  manned  aircraft 
development  is  therefore  partly  allayed  by  the  large  number  of  unmanned  aircraft 
currently  being  developed.  This  pattern  bodes  poorly  for  innovation  and  growth  in 
manned  aircraft,  but  indicates  the  future  of  UAV’s  is  likely  to  be  marked  with  further 
breakthrough  capabilities  and  discoveries.  Ultimately,  the  question  is  not  whether  there  is 
a  body  in  the  cockpit.  The  question  is  whether  the  system  provides  a  dominant  capability 
over  our  adversaries,  and  the  UAV  community  is  well  on  its  way  to  demonstrating 
exactly  that. 
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There  are  several  reasons  UAV  developers  are  able  to  provide  such  an  impressive 
variety  of  platfonns.  The  most  obvious  is  the  immature  state  of  UAV  applications  -  it  is 
still  a  relatively  new  field,  with  many  niches  unfilled  and  opportunities  unmet.  Unlike  the 
more  mature  manned  fighter  or  bomber  communities  (for  example),  we  are  still  learning 
what  UAV’s  can  do  and  have  not  yet  fully  established  their  operational  boundaries,  or 
even  the  optimal  doctrines  to  guide  their  use.  A  cursory  survey  of  UAV’s  currently  in 
operation  reveals  a  wide  variety  of  body  shapes,  sizes,  missions,  payloads,  endurances 
and  capabilities  (see  for  example  the  UAV’s  depicted  in  Figure  6).  This  diversity  is  made 
possible  because  the  underlying  technology  is  relatively  low-cost,  simple  and  can  be 
rapidly  developed  and  deployed  by  small  teams.  This  also  fosters  competition  among 
developers,  which  has  the  potential  to  convey  significant  benefits  for  the  DoD  in  tenns  of 
cost,  availability  and  performance. 

The  aspect  of  UAV  development  most  relevant  to  this  research  is  the  fact  that 
UAV’s  are,  perhaps  uniquely  in  the  history  of  military  technology  development, 
inherently  FISTy.  That  is,  when  compared  to  previous  systems  designed  for  this  mission 
area  (i.e.  manned  aircraft),  modem  UAV’s  are  developed  on  shorter  timelines  and  smaller 
budgets,  are  less  complex  and  are  much  smaller.  Individual  UAV’s  may  currently  be  less 
capable,  but  since  the  cost  to  develop,  operate  and  maintain  a  UAV  is  a  fraction  of  a 
manned  aircraft,  they  can  make  up  that  individual  operational  shortfall  by  deploying  in 
larger  quantities  and  with  more  diversity.  A  sufficiently  large  fleet  of  UAV’s  can 
outperfonn  manned  ISR  aircraft  (for  example),  while  still  costing  much  less.  The 
inherently  FISTy  nature  of  UAV  technology  is  a  primary  reason  for  the  operational 
effectiveness  of  UAV’s  in  Iraq  and  Afghanistan. 
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For  example,  the  Army’s  Shadow  UAV  has  a  gross  weight  of  375  pounds,  a  12’ 
wingspan  and  a  $275K  pricetag  (Shadow  200  Factsheet,  GlobalSecurity.org).  The  Marine 
Corps’  Dragon  Eye  UAV  is  even  smaller,  simpler  and  less  expensive.  It  weighs  in  at  a 
mere  five  pounds,  can  be  assembled  in  the  field  in  ten  minutes  and  launched  by  hand. 
Training  a  Dragon  Eye  operator  takes  less  than  a  week,  and  $100,000  buys  an  entire 
system,  including  three  air  vehicles  (Barnard  Microsystems  website).  In  tenns  of 
capability,  the  Dragon  Eye  provides  “direct  threat  reduction  and  reconnaissance”  to  small 
Marine  units  (Dragon  Eye  Factsheet,  GlobalSecurity.org). 

FIST  Fosters  “Luck” 

Despite  the  predictions  and  prognostications  of  various  analysts  (myself  included) 
it  is  reasonable  to  assert  that  program  outcomes  are  to  a  certain  extent  the  result  of 
unknown,  unpredictable,  unmeasurable  phenomena  and  hidden  forces,  which  for 
convenience  we  can  refer  to  as  “luck.”  Given  the  centrality  of  luck  in  determining 
program  outcomes,  one  might  wonder  whether  there  is  a  way  for  project  leaders  to 
become  luckier. 

Professor  Richard  Wiseman  studied  the  phenomenon  of  “luck  perception  and  luck 
production”  over  a  period  of  ten  years  (Wiseman,  2003)  He  concluded  that  luck,  properly 
defined,  is  real  and  can  be  created.  Specifically,  his  research  shows  that  luck  perception 
and  luck  production  are  both  related  to  personality  factors  such  as  optimism, 
extroversion,  openness,  and  low  levels  of  anxiety  (Wiseman,  2003).  To  the  extent  that 
such  attributes  can  be  learned  and  practiced,  a  person  can  essentially  manufacture  their 
own  luck. 
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My  colleague  Maj  Chris  Quaid  and  I  examined  Wiseman’s  research  in  an  article 

titled  “Courage,  Judgment  and  Luck,”  writing: 

Wiseman’s  research  showed  that  while  people  who  describe  themselves  as  lucky 
are  not  more  likely  to  win  the  lottery,  they  are  more  likely  to  experience  positive 
outcomes  in  other,  less  random  activities.  For  example,  a  person’s  extroversion 
creates  a  large  social  network,  which  can  lead  to  ‘fortuitous’  connections  with 
people  and  resources.  Openness  to  new  experiences  leads  to  action,  and  as  John 
Nash  said  in  A  Beautiful  Mind,  “The  probability  of  my  success  increases  with 
every  attempt.”  (Quaid  &  Ward  2007:29) 

The  FIST  approach  tends  to  encourage,  attract,  and  reinforce  the  personality 
factors  Dr.  Wiseman  identified  as  components  of  luck.  Starting  a  program  with  a  short 
schedule  and  a  tight  budget  requires  optimism.  Replacing  paperwork  with  teamwork  and 
formal  reviews  with  informal  trust  (i.e.  simplifying)  requires  openness,  low  levels  of 
anxiety  and  a  certain  amount  of  extroversion.  The  same  goes  for  the  FIST  emphasis  on 
talent  over  process. 

Taleb  made  a  similar  observation  to  Wiseman,  pointing  out  it  is  possible  to 
“create  luck  by  sheer  exposure”  (Taleb,  2007:170).  Because  the  FIST  approach  involves 
a  large  number  of  small,  rapid  developments  (made  possible  by  lower  costs  and  short 
schedules),  the  rate  of  “sheer  exposure”  is  much  higher  than  in  the  traditional  approach, 
giving  project  leaders  greater  exposure  to  “the  opportunity  to  be  lucky.”  (Taleb, 
2007:222). 

Like  any  effort,  FIST  is  most  likely  to  be  successfully  implemented  by  lucky 
program  managers  than  unlucky  ones.  However,  FIST  leads  to  luck  production  by 
fostering  openness  and  extroversion  and  increasing  the  exposure  rate  of  new  ideas  and 
technologies  through  rapidly  designing,  fielding  and  operating  new  systems.  The 
traditional  approach,  in  contrast,  relies  on  small  numbers  of  big,  expensive,  tightly- 
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controlled,  decades-long  development  projects,  which  minimizes  the  possibility  of 
fortuitous  discoveries  and  thus  thwarts  luck. 

Keys  To  Implementation 

There  are  a  number  of  things  project  leaders  can  do  to  implement  the  FIST 
approach,  beginning  with  a  recognition  that  FIST  is  one  unified  idea,  not  four  separate 
ideas.  The  four  components  of  FIST  are  connected  in  many  ways,  but  most  prominently 
by  a  common  thread  of  simplicity.  Accordingly,  project  leaders  should  emphasize  that 
value  in  particular.  Second,  project  leaders  need  to  genuinely  understand  the  values  and 
avoid  settling  for  ineffective  imitations  such  as  the  mere  appearance  of  speed  instead  of 
an  actual  emphasis  on  rapid  delivery,  or  simplistic  solutions  instead  of  a  more  profound 
simplicity.  FIST  also  requires  a  certain  tolerance  for  failure,  so  project  leaders  should 
expect  to  pursue  multiple  iterations.  Finally,  the  FIST  value  set  is  most  effective  if  the 
values  are  shared  by  all  the  stakeholders,  so  project  leaders  would  do  well  to  discuss  the 
concept  with  their  team,  their  customers  and  their  suppliers  alike. 

FIST  is  One  Idea,  Not  Four 

As  mentioned  in  the  introduction,  the  components  of  the  FIST  value  set  cannot  be 
isolated  without  degrading  their  ability  to  contribute  to  the  outcome.  Successful 
application  of  the  FIST  approach  requires  an  appreciation  of  FIST  as  a  single  idea,  with 
four  internally-consistent  and  mutually-reinforcing  elements,  not  as  a  series  of 
independent  alternatives  for  project  leaders  to  weigh  against  each  other  in  trade-off 
analyses.  This  runs  counter  to  the  prevailing  sense  that  project  leaders  must  “pick  two” 
from  the  list  of  cost,  schedule  and  performance,  sacrificing  one  to  deliver  the  others.  The 
FIST  approach  suggests  that  project  leaders  should  “pick  all  three.”  Indeed,  project 
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leaders  must  pick  all  three  if  the  scheme  is  to  work.  An  attempt  to  shorten  the  schedule 
without  reducing  the  budget,  for  example,  is  likely  to  result  in  schedule  overruns,  which 
in  turn  drive  cost  overruns  -  a  vicious  circle  not  easily  broken. 

Emphasis  on  Simplicity 

One  explanation  for  the  seeming  validity  of  the  popular  “pick  two”  heuristic  is  a 
failure  to  address  simplicity.  Attempts  to  simultaneously  improve  cost,  schedule  and 
perfonnance  without  reducing  complexity  tends  to  lead  to  failure,  as  NASA  discovered  in 
the  late  1990’s.  Excessive  complexity  in  the  organization  and  the  system  virtually 
requires  project  leaders  to  improve  only  two  sides  of  the  “Program  Manager’s  Iron 
Triangle,”  while  simple  organizations  can  produce  simple  technologies  that  are 
simultaneously  faster,  better  and  cheaper. 

The  A- 10  is  a  prime  example  of  a  system  developed  with  a  healthy  appreciation 
for  the  central  role  of  simplicity.  The  project  leaders  deliberately  pursued  a  simple 
approach  to  what  was  then  called  the  A-X,  and  “...  it  was  intended  that  simplicity  of 
design  would  lead  to  a  shorter  development  time,  lower  life  cycle  cost,  reduced 
maintenance  times,  increased  sortie  rates  and  the  ability  to  operate  from  austere  bases.” 
(Jacques  &  Strouble,  2008:22) 

Simplicity  also  appears  to  have  a  strong  direct  correlation  with  operational 
availability.  As  Jacques  and  Strouble  point  out,  “...  there  was  an  observed  ratio  of  3:1  in 
MMH/FH  [Maintenance  Man  Hours  per  Flying  Hour]  between  the  most  complex  and  the 
simplest  strike  aircraft.”  (Jacques  &  Strouble,  2008:23)  That  is,  complex  aircraft  required 
more  maintenance  per  flight  hour  than  simple  aircraft  do,  which  leads  to  increased 
availability  and  decreased  costs  for  simpler  aircraft. 
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The  National  Reconnaissance  Office’s  FIA  project  provides  a  contrary  example, 
and  illustrates  the  danger  inherent  in  pursuing  a  portion  of  the  FIST  value  set  in  isolation 
rather  than  incorporating  all  four  components.  Ansehno  &  Butler  observe  that  “FIA... 
was  procured  under  the  faster-better-cheaper  mindset  of  the  1990’s,”  (Ansehno  &  Butler, 
2005:2).  Early  reports  describe  the  project  in  terms  of  using  smaller,  cheaper  satellites,  a 
concept  that  project  leaders  never  apparently  bought  in  to  and  which  they  quickly 
abandoned.  While  FIA  was  ostensibly  an  attempt  to  apply  the  Faster,  Better,  Cheaper 
approach,  project  leaders  were  clearly  committed  from  the  start  to  developing  a  system 
that  was  tremendously  complex.  The  failure  to  reduce  complexity  lead  to  the  project’s 
eventual  (one  might  say  inevitable)  failure. 

The  danger  introduced  by  complexity  was  not  unforeseen,  as  the  following 
references  illustrate: 

“. . .  an  internal  assessment  that  questioned  whether  its  lofty  technological  goals 
were  attainable  given  the  tight  budget  and  schedule.”  (Taubman,  2007:2) 

“The  satellites  were  loaded  with  intelligence  collection  requirements,  as 
numerous  intelligence  and  military  services  competed  to  influence  the  design. 
Boeing’s  initial  design  for  the  optical  system  that  was  the  heart  of  one  of  the  two 
new  satellite  systems  was  so  elaborate  that  optical  engineers  working  on  the 
project  said  it  could  not  be  built.”  (Taubman,  2007:2) 

“...  outside  engineers  questioned  the  photo  satellite’s  intricate  optical  system...  it 
soon  became  clear  the  system  could  not  be  built.”  (Taubman,  2007:8-9) 

“. . .  the  government’s  addition  of  more  stringent  requirements,  the  new  satellites 
are  looking  more  like  the  ‘Battlestar  Galacticas’  they  were  supposed  to  replace 
than  the  simple  spacecraft  envisioned  in  1999.”  (Ansehno  &  Butler,  2005:1) 

The  initial  desire  to  be  “faster,  better,  cheaper,”  was  directly  undercut  by  the 
preference  for  complexity  on  the  part  of  both  the  government  and  contractors.  By 
pursuing  complexity  instead  of  simplicity,  project  leaders  surrendered  any  benefits  that 
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might  have  been  conveyed  by  the  Fast  or  Inexpensive  values.  They  also  forsook  the 
ability  to  express  the  Tiny  value. 

The  cover  story  in  Time  Magazine’s  March  7,  1983  issue  paints  a  similar  picture. 
In  this  report,  a  preference  for  complexity  gets  much  of  the  blame  for  a  variety  of  failed 
military  technology  projects.  Quoting  a  Heritage  Foundation  report,  the  article  points  out 
“The  evidence  suggests  that  complex  technology  is  usually  relatively  ineffective.” 
(Isaacson,  1983:4).  It  also  cites  Spinney’s  1980  Defense  Facts  of  Life  report,  which  it 
says  “argued  that  the  pursuit  of  complex  technology  has  resulted  in  the  production  of 
weapons  that  are  high  in  cost,  few  in  number  and  questionable  in  effectiveness.” 
(Isaacson,  1983:3). 

The  article  supports  the  position  of  reformers  who  “argue  that  in  many  cases  the 
simpler  weapons  are  actually  more  effective.”  (Isaacson,  1983:7).  The  article  goes  on  to 
suggest  that  “whether  a  weapon  can  be  afforded  in  adequate  numbers  should  be  a  more 
important  concern  than  whether  it  is  state-of-the-art...”  (Isaacson,  1983:12).  This  aligns 
with  the  idea  that  the  FIST  values  must  be  used  in  concert,  and  cannot  be  implemented 
piecemeal. 

Use  Mature  Technologies 

One  of  the  common  themes  among  FIST  projects  is  the  use  of  mature 
technologies.  This  suggests  that  where  mature  technology  is  not  available,  the  FIST 
approach  may  not  be  advisable.  Alternately,  one  may  posit  that  mature  technology  is 
almost  always  available,  and  project  leaders  need  only  resist  the  temptation  to  stake  their 
outcome  on  a  hoped-for-but-currently-unavailable  future  technology. 
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Throughout  this  research,  a  preference  for  mature  technology  is  most  often  linked 
to  the  Simple  value,  but  because  the  FIST  values  are  interrelated,  using  mature 
technologies  can  also  be  an  expression  of  the  Fast,  Inexpensive  and  Tiny  values. 

When  it  is  important  and  good  for  a  project  to  be  Fast  and  Inexpensive,  mature 
technologies  are  an  obvious  choice.  The  costs  associated  with  mature  technologies  are 
more  likely  to  be  known  and  documented  than  those  associated  with  new  technologies, 
making  cost  estimations  easier  and  more  reliable.  Using  proven  technologies  can  also 
save  time  and  money  by  allowing  project  leaders  to  minimize  R&D,  testing  and 
experimentation. 

The  XP-75  Eagle  and  the  DIVAD  programs  both  illustrate  some  of  the  pitfalls 
inherent  in  using  mature  technology.  The  DIVAD’s  selection  of  inappropriate 
components  has  already  been  discussed.  The  XP-75  did  much  the  same  thing,  and 
encountered  very  similar  issues.  Many  of  the  off-the-shelf  components  were  found  to  be 
inadequate  once  they  were  integrated  into  the  airframe,  and  required  a  significant  amount 
of  rework.  This  greatly  reduced  the  anticipated  benefit  of  using  existing  components,  as 
the  following  quotes  show: 

“By  enlarging  the  tail  surface,  most  of  the  apparent  instability  could  be  overcome, 
but  this  meant  it  would  no  longer  be  possible  to  use  the  ready-built,  off-the-shelf 
stock  design  already  in  production...”  (Holley,  1987:590) 

“To  make  matters  worse,  the  Allison  engine,  which  had  been  rushed  into 
production  before  it  was  thoroughly  debugged,  was  not  perfonning  up  to 
expectations...”  (Holley,  1987:590) 

“The  remedy,  eventually  devised,  was  to  extend  the  ailerons  out  to  the  wingtips  to 
insure  greater  stability  and  control.  But  this,  too,  required  reworking  the  standard 
ready-built  wings. . .”  (Holley,  1987:590) 

“. . .  more  and  more  evidence  emerged  to  indict  the  whole  scheme  to  use  off-the- 
shelf  stock  components  to  build  a  superior  high-performance  aircraft.”  (Holley, 
1987:591) 
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Recall  from  the  earlier  discussion  on  the  Fast  value  that  valuing  speed  does  not 
dictate  a  specific  development  timeline.  Thus,  when  a  project  leader  borrows  technology 
from  a  project  that  has  been  in  development  for  20  years,  at  a  cost  of  billions  of  dollars, 
this  does  not  disprove  the  presence  of  the  Fast  and  Inexpensive  values  for  that  system.  In 
fact,  leveraging  old  money  and  existing  technology  to  build  a  new  system  rather  than 
spending  new  money  to  develop  new  technology  is  often  an  explicit  expression  of  the 
FIST  values. 

A  common  short  cut  for  delivering  operationally  effective  systems  is  to  “start 
before  you  start,”  taking  advantage  of  existing  research,  knowledge,  technologies  and 
systems,  and  integrating  them  in  new  and  interesting  ways  to  deliver  the  needed 
capabilities.  Technological  breakthroughs  and  scientific  discoveries  often  take  decades  to 
accomplish  and  can  rarely  be  accomplished  according  to  a  program  manager’s  schedule. 
Thus,  it  makes  sense  to  rely  on  breakthroughs  and  discoveries  that  have  already  been 
made,  rather  than  hope  one  will  be  made  in  time  to  be  useful.  In  fact,  the  secret  behind 
most  of  the  “airplane  designed  in  a  hotel  room  over  the  weekend”  stories  (e.g.  the  P-75, 
B-52,  etc)  is  that  the  designers  were  not  starting  from  scratch. 

Avoid  Imitations 

FIST  works  best  when  the  FIST  values  are  deeply  understood  and  held,  not 
treated  as  window  dressing  or  a  simplistic  quick- fix.  Like  most  approaches  to 
organizational  behavior,  the  FIST  approach  can  be  easily  undennined  by  project  leaders 
who  are  not  committed  to  the  principles  involved  or  who  do  not  fully  understand  them. 
The  Army’s  DIVAD  program  is  an  excellent  example  of  project  leaders  settling  for  a 
superficial  appearance  of  the  FIST  values,  rather  than  actually  expressing  them. 
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At  first  glance,  DIVAD  had  all  the  appearances  of  a  FIST  project,  with  frequent 
assertions  in  support  of  several  of  the  FIST  values  and  many  instances  of  contractual, 
procedural  and  technical  decisions  being  made  based  on  a  desire  to  cut  cost  and  schedule. 
For  example,  the  contractor  was  given  the  flexibility  to  trade-off  requirements  for  cost 
and  schedule  savings,  which  seems  to  indicate  that  Fast  and  Inexpensive  mattered  more 
than  high-technology  or  performance.  However,  upon  closer  examination,  this  project  is 
revealed  to  be  a  pseudo-FIST  effort,  with  a  significant  emphasis  on  appearing  to  be  fast 
and  inexpensive,  rather  than  actually  embracing  or  understanding  how  to  apply  these 
values. 

One  commentator  pointed  that  “...  greater  priority  has  been  given  to  adhering  to 
the  schedule  than  to  correcting  some  serious  system  performance  problems  at  this  time.” 
(Ditton,  1988:8).  Another  observed  that  “...calendar  schedule  compliance  had  a  higher 
priority  than  test  matrix  completion...  the  combination  of  these  factors  precluded 
complete  execution  of  the  test  plan.”  (Adam,  1987:33) 

The  project  leaders’  determination  to  avoid  a  schedule  slip  led  them  to  ignore 
serious  system  performance  problems.  While  the  team  achieved  a  high  degree  of 
“ calendar  schedule  compliance,”  one  might  ask  what  it  means  to  be  on  schedule  if  the 
system  does  not  actually  do  what  it  needs  to  do  by  the  delivery  date.  A  project  that  fails 
to  meet  the  critical  performance  requirements  by  the  necessary  date  cannot  be  considered 
“on  schedule”  in  anything  more  than  a  superficial  sense. 

Similarly,  the  literature  asserts  that  project  leaders  “successfully  controlled  costs” 
on  the  DIVAD  development,  but  this  is  not  an  accurate  statement.  Project  leaders  merely 
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controlled  spending,  not  cost.  That  is,  they  spent  as  much  as  they’d  planned  to  spend  in 
the  time  allotted,  but  they  purchased  far  less  than  their  customers  needed,  and  the 
DIVAD’s  performance  suffered  accordingly.  By  purchasing  less,  they  were  not  genuinely 
expressing  the  Inexpensive  value.  They  simply  settled  for  the  illusion  of  Inexpensive, 
with  disastrous  results.  The  truth  is,  they  paid  a  lot  and  got  very  little,  in  large  part 
because  their  decisions  were  not  consistent  with  the  Inexpensive  value. 

Much  is  made  of  the  Anny’s  decision  to  simplify  the  procurement  process, 
freeing  Ford  from  the  complexities  of  government  regulation  and  requirements.  However, 
the  Army’s  “hands  off’  policy  was  unfortunately  accompanied  by  an  “eyes  shut  and  ears 
plugged”  approach,  creating  the  appearance  of  simplifying  the  situation  in  the  short  term, 
while  actually  making  things  more  complicated  down  the  road.  There  is  a  difference 
between  not  interfering  with  the  contractor’s  decisions  and  not  really  being  aware  of 
them,  a  distinction  the  DIVAD  project  leaders  did  not  seem  to  grasp.  Simply  handing  a 
contract  to  a  company  and  asking  them  to  report  back  when  the  job  is  complete  is  not 
simple  -  it  is  simplistic. 

Further  complicating  things  was  the  DIVAD’s  attempt  to  use  mature  technologies. 
Ditton  explains  “...  the  DIVAD  gun  system  was  supposed  to  be  an  integration  of  proven 
major  components,  including  the  M-48A5  tank  chassis,  twin  Swedish  Bofors  40 
millimeter  guns,  and  radars  from  the  F-16  fighter.”  (Ditton,  1988:4)  This  is  only  an 
expression  of  the  Simple  value  if  project  leaders  select  mature  technologies  that  actually 
perfonn  the  required  functions.  Sadly,  the  DIVAD  project  leaders  made  several 
unfortunate  selections. 
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“...Although  each  major  subsystem  was  a  proven  component,  the  sum  of  the 
components  could  not  match  contract  requirements,  much  less  battlefield  reality. 
The  F-16  radar  operates  on  detection  of  movement  and  was  successful  at 
acquiring  moving  targets.  Unfortunately,  it  had  difficulty  acquiring  stationary 
targets.”  (Ditton,  1988:6) 

Thus,  the  decision  to  incorporate  mature  technology  had  the  appearance  of 
simplicity,  but  not  the  substance.  The  F-16  radar  was  only  good  at  detecting  moving 
targets,  while  the  DIVAD  needed  to  acquire  stationary  ones.  Similarly,  the  DIVAD  was 
built  on  an  existing  tank  chassis,  which  would  have  made  sense  if  that  chassis  had  been 
powerful  enough  to  haul  the  DIVAD’s  full  weight.  In  fact,  “the  M-48’s  750-horsepower 
engine  was  designed  to  power  a  50-ton  tank,  not  the  60-ton  Sergeant  York  vehicle.” 
(Adam,  1987:32)  Since  the  DIVAD’s  weight  exceeded  the  engine’s  designed  capacity  by 
20%,  one  might  argue  that  the  M-48,  while  mature,  for  this  purpose  was  neither  proven 
nor  appropriate.  The  Secretary  of  Defense  canceled  the  DIVAD  project  on  August  27, 
1985. 

The  lesson  from  the  DIVAD  is  clear  -  superficial  expressions  of  the  FIST  values 
are  not  adequate.  There  is  a  significant  difference  between  being  Fast,  Inexpensive  and 
Simple,  and  merely  being  Hasty,  Cheap  and  Simplistic.  Adhering  to  a  project’s  schedule 
or  budget  is  meaningless  if  the  necessary  requirements  are  not  satisfied  by  the 
expenditure,  and  mature  technologies  are  only  good  if  they  perform  the  required 
functions.  In  order  to  genuinely  use  the  FIST  approach,  project  leaders  must  deeply 
understand  the  value  set  and  how  to  properly  express  it. 
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Iterate  (aka  Expect,  Understand  and  Value  Failures) 

The  FIST  approach  is  most  successful  when  done  iteratively;  it  is  most  risky 

when  done  as  a  one-shot  deal.  The  development  of  the  XP-75  Eagle  during  World  War  II 

clearly  illustrates  this  principle: 

“In  September  1941,  the  corporation  [General  Motors]  approached  Maj  Gen  O.P. 
Echols,  the  commanding  general  of  Materiel  Command,  with  a  proposal  to 
develop  a  fighter  aircraft  in  a  remarkably  short  time  by  using,  as  far  as  possible, 
structures,  controls,  and  accessories  already  in  full  production  for  other  aircraft  in 
order  to  obviate  the  need  for  long  delays  in  tooling  up  for  production.”  (Holley, 
1987:587) 

General  Arnold,  the  commanding  general  of  the  Anny  Air  Forces,  quickly  made  it 
clear  that  the  project  leaders  had  to  get  things  right  the  first  time,  demanding  that  the 
initial  experiment  meet  the  full  requirements.  ‘“If  it  does  not  meet  our  requirements,’  he 
wrote,  ‘all  orders  may  be  cancelled;  everyone  must  understand  this.’”  (Holley,  1987:589) 
As  it  turned  out,  the  initial  experiment  showed  a  lot  of  promise,  but  was  not  a 
complete  success.  Accordingly,  “On  October  4,  1944,  the  chief  of  air  staff,  Lt.  Gen.  B.M. 
Giles,  signed  the  order  terminating  the  contract.”  (Holley,  1987:591)  It  is  unknown 
whether  subsequent  attempts  would  have  led  to  better  outcomes,  but  we  cannot  rule  out 
the  possibility. 

Expecting  the  initial  version  of  experimental  aircraft  to  perfonn  well  and  meet  all 
the  requirements  is  not  consistent  with  the  FIST  approach.  This  preference  for  immediate 
perfection  raises  the  importance  of  perfonnance  ahead  of  timeline  or  budget,  thus 
diminishing  the  role  of  the  FIST  values.  It  also  overlooks  the  fact  that  each  attempt  is 
made  with  minimal  exposure  to  loss.  Using  FIST,  project  leaders  should  plan  on  iterative 
deliveries  of  systems,  with  increasing  degrees  of  capability  and  reliability  in  each 
iteration. 


95 


Establish  Shared  Values 

The  FIST  approach  to  system  development  is  most  effective  when  the  developers 
and  the  customers  are  operating  from  a  shared  set  of  values,  and  are  in  agreement  as  to 
the  measures  of  merit  for  the  proposed  system.  Northrop  learned  this  lesson  painfully  on 
the  F-20  Tigershark  project. 

Following  their  success  with  the  F-5,  Northrop  explicitly  focused  on  building  a 
FISTy  aircraft:  the  F-20  Tigershark.  It  was  intended  for  sale  to  foreign  nations,  and  was 
designed  to  be  an  appealing  alternative  to  the  downgraded  versions  of  the  F-16  Falcon 
approved  for  foreign  sale  under  the  Carter  administration’s  policies. 

In  1981,  the  Reagan  administration  changed  the  policy,  forcing  the  F-20  to 
compete  against  the  fully-configured  F-16,  instead  of  against  the  degraded  version.  As 
soon  as  foreign  buyers  had  the  option  to  purchase  the  F-16,  the  F-20’s  appeal  was  much 
diminished. 

Martin  &  Schmidt  point  out,  “Now  that  F-16A’s  had  been  sold  to  Venezuela  and 
Pakistan,  most  developing  countries  began  requesting  similar  aircraft,  regardless  of  their 
budgets  or  realistic  assessments  of  their  security  threats.”  (Martin  &  Schmidt,  1987:14- 
15).  Rather  than  base  their  purchase  decision  on  an  assessment  of  the  actual  threat 
environments  they  faced,  foreign  governments  preferred  the  larger,  more  expensive,  more 
complex  F-16,  and  they  “demanded  the  highest  level  of  technology  they  could  acquire.” 
(Martin  &  Schmidt,  1987:21). 

This  was  a  psychological  and  emotional  decision  on  the  part  of  foreign 
governments,  based  on  a  desire  to  appear  fully  modem  and  tough.  Rather  than  valuing  the 
F-20’s  simplicity,  availability  and  low  cost,  they  preferred  the  F-16’s  relative  complexity, 
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higher  costs  and  greater  performance.  It  should  be  pointed  out  that  much  the  same 
decision  process  goes  on  in  the  US  as  well,  so  they  were  only  following  the  DoD’s 
example.  In  the  end,  the  very  values  which  led  to  the  development  of  the  Tigershark  as  a 
simple,  reliable,  low-cost  fighter  jet  actually  undercut  its  marketability,  because  these 
values  were  not  shared  by  the  potential  customers. 

Northrop  was  not  able  to  sell  even  a  single  F-20.  The  Tigershark’s  limited 
performance  was  certainly  a  contributing  factor,  as  was  the  change  in  US  policy  on 
weapon  exports.  However,  the  primary  reason  appears  to  be  the  striking  mismatch 
between  Northrop’s  design  values  and  the  values  of  its  potential  clients. 

FIST  Values  /  American  Values 

Value  statements  are  cultural  artifacts,  and  the  American  defense  community  has 
a  well  established  set  of  values  that  help  to  define  its  culture.  Tobias  et  al  point  out  that 
“Most  Americans  subscribe  to  the  notion  that  ‘You  get  what  you  pay  for’;  that  the  more 
expensive  a  weapon  is,  the  better  it  is.  If  it  can  do  more — fly  higher  or  drive  faster — it 
must  be  better.  The  Pentagon,  too,  is  inclined  in  this  direction.”  (Tobias  et  al,  1982:366). 

Thus,  it  might  seem  the  FIST  value  set,  with  its  emphasis  on  small,  simple  and 
inexpensive  technologies,  is  fundamentally  at  odds  with  America’s  and  the  DoD’s 
cultural  values.  That  is  apparently  often  the  case,  as  Tobias  et  al  pointed  out:  “What  does 
our  weaponry  and  style  of  warfare  say  about  us?  As  we  have  already  noted,  we  often 
build  weapons  that  are  on  the  frontiers  of  their  technology:  complex,  sophisticated  and 
expensive...  [and  we]  view  technological  innovation  as  the  decisive  factor  in  victory.” 
(Tobias  et  al,  1982:382).  The  authors  also  point  out  the  DoD’s  “goal  is  to  maximize 
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performance,  often  with  little  regard  for  cost,”  (Tobias  et  al,  1982:260),  then  further 

highlight  the  distance  between  the  DoD’s  values  and  the  FIST  values: 

We  are  a  society  that  places  a  high  value  on  the  lives  of  our  soldiers  and  we 
would  rather  expend  equipment  and  materiel,  however  costly,  than  lives.  In 
Vietnam  our  lavish  use  of  exotic  weapons,  airpower,  and  firepower  reflected  this 
and  contributed  mightily  to  the  dollar  cost  of  the  war...  we  design  and  build 
weapons  with  an  eye  toward  the  creature  comforts  of  their  crews.  This  partly 
accounts  for  the  larger  size  and  higher  cost  of  our  warships  and  tanks,  for  example, 
in  comparison  to  those  of  the  Soviets,  who  give  physical  comfort  a  much  lower 
priority.  (Tobias  et  al,  1982:383) 

In  other  words,  according  to  Tobias  et  al,  the  DoD  considers  many  other  factors, 
including  physical  comfort,  to  be  more  important  than  low-cost,  rapid  development, 
simplicity  or  smallness.  While  Tobias  et  al  were  writing  in  1982,  little  has  changed  in  the 
DoD’s  value  structure  since  that  time. 

However,  it  is  not  clear  the  American  pursuit  of  big,  expensive,  complex  and 
comfortable  systems  actually  enhances  survivability,  reliability,  combat  effectiveness, 
availability  or  other  measures  of  operational  outcome.  In  fact,  these  traditional,  non-FIST 
values  might  actually  put  more  American  lives  at  risk,  not  to  mention  our  overall  defense 
posture,  military  capabilities  and  national  economic  health.  Tobias  et  al  point  out,  “...  we 
get  the  weapons  we  deserve:  the  product  of  our  infatuation  with  gadgets,  our  optimism, 
and  our  extravagance.  But  in  another  sense  we  may  not  be  getting  the  weapons  we  need.” 
(Tobias  et  al,  1982:383) 

While  this  research  turned  up  many  examples  of  big,  expensive,  complex  systems 
that  performed  well,  it  is  not  clear  the  non-FIST  approach  leads  to  optimal  outcomes.  It  is 
entirely  possible  that  equivalent  perfonnance  could  have  been  achieved  on  those  projects 
with  a  smaller  investment  of  time  and  money,  had  such  an  approach  been  attempted.  It  is 
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also  observed  that  large  budgets  and  endless  schedules  lead  to  negative  outcomes  with 
depressing  regularity. 

Tobias  et  al  also  point  out  that  when  we  examine  the  history  of  weapons  and 
warfare,  we  “find  lots  of  examples  of  weapons  that  were  cheap  and  successful,  ‘winners,’ 
and  others  that  were  expensive  and  unsuccessful,  ‘losers.’”  (Tobias  et  al,  1982:366).  In 
the  27  years  since  their  book  was  published,  the  cheap  winner  /  expensive  loser  pattern 
has  clearly  continued. 

Interestingly,  Tobias  et  al  also  observe  that  America’s  “reliance  on  technology  is 
relatively  new,  by  and  large  a  post-World  War  II  phenomenon.  In  the  past,  we  relied 
instead  on  large  quantities  of  dependable  but  conservatively  designed  weapons  that  could 
be  operated  by  quickly  trained  troops.”  (Tobias  et  al,  1982:382).  This  description  of 
American  design  values  lines  up  closely  with  the  FIST  model  and  echoes  the  values 
evident  in  the  Soviet  T-34  tank,  which  performed  so  admirably  against  the  German 
Panzer  IV’s  in  World  War  II.  Indeed,  America’s  pre-World  War  II  era,  with  its  emphasis 
on  FIST  design  values,  has  an  impressive  track  record  of  operational  effectiveness. 

The  pre-World  War  II  era  was  not  the  last  group  of  Americans  to  embrace  the 
FIST  values.  As  my  research  data  shows,  small  groups,  at  many  times  throughout  recent 
history,  have  accepted  and  applied  these  values  to  great  effect. 

In  the  1970’s  and  1980’s,  a  group  known  as  The  Fighter  Mafia  led  the 
development  of  several  aircraft,  including  the  F-16  and  A- 10.  As  Burton  explains,  the 
Figher  Mafia’s  “philosophy  is  easily  stated:  Technology  can  be  used  to  produce  simpler, 
hence  lower-cost  weapons;  lower-cost  weapons  can  pennit  larger  forces...  The  reformers 
were  concerned  about  complexity,  cost  and  combat  usefulness,  not  ‘high’  or  Tow’ 
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technology.”  (Burton  1993:  26-27).  In  discussing  the  development  of  the  A- 10,  Burton 

writes  “The  reformers  preferred  eyeballs  as  primary  sensors  in  finding  tanks  and  the  use 

of  inexpensive  cannon  in  killing  them.”  (Burton  1993:26) 

Col  James  Burton,  one  of  the  Fighter  Mafia  refonners,  even  proposed  the  Air 

Force  develop  a  new  airplane,  which  he  named  the  Blitzfighter.  He  writes: 

I  prepared  an  advocacy  briefing  that  called  for  the  development  of  a  small,  simple, 
lethal,  and  relatively  cheap  airplane  that  would  be  designed  solely  for  close 
support  of  the  ground  troops. . .  I  wanted  an  airplane  in  the  5,000-  to  10,000-pound 
class  (one -tenth  the  weight  of  the  Enhanced  Tactical  Fighter),  one  smaller  than 
any  combat  airplane  in  the  inventory  (one-fourth  the  size  of  the  A- 10),  and  one 
that  cost  less  than  $2  million.  At  this  price,  we  could  flood  the  battlefield  with 
swarms  of  airplanes. 

The  airplane  would  be  designed  around  a  four-barrel  version  of  the  same  cannon 
that  was  in  production  on  the  A- 10,  which  used  a  seven-barrel  cannon  that  fired 
shells  costing  only  $13  apiece...  The  Blitzfighter  would  have  no  high-tech  bells 
and  whistles  and  no  wonder  weapons...  With  the  ability  to  operate  from  grass 
fields,  the  Blitzfighter  did  not  demand  fixed,  expensive  airfields  that  probably 
would  cease  to  exist  ten  minutes  after  a  war  started. . .  (Burton  1993:57-58) 


This  concept  is  a  clear  sign  of  the  FIST  values  at  work.  Burton’s  envisioned 
Blitzfighter  was  never  built,  apparently  because  Burton’s  superiors  did  not  share  his 
values — which  is  one  of  the  pitfalls  inherent  in  attempting  to  use  the  FIST  values.  When 
they  are  not  shared  by  one’s  customers  or  superiors,  the  result  is  seldom  acceptable  or 
successful.  Northrop  learned  this  lesson  the  hard  way  on  the  F-20  program,  and  Burton 
learned  it  many  times  over. 
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The  FIST  Heuristics 

...there  need  to  be  more  general  heuristics  to  help  guide  those  involved  in  this 

growth  industry  [systems  architecting] .  (Wieser,  et  al,  2006:44). 

Program  management  and  systems  engineering  are  largely  social  disciplines,  not 
hard  sciences.  They  involve  subjective  questions  of  morale,  opinion,  values  and  other 
intangibles,  and  their  secrets  are  generally  not  revealed  through  statistical  analysis, 
however  complex.  Attempts  to  treat  system  development  methodologies  as  optimizable 
mathematical  operations  are  inappropriate  and  tend  to  produce  unhelpful  solutions  or 
irrelevant  insights,  based  on  an  illusion  of  rationality. 

To  borrow  a  phrase  from  Nicholas  Taleb,  the  mathematical  approach  treats 
program  management  like  “a  second-rate  engineering  problem  for  those  who  want  to 
pretend  that  they  are  in  the  physics  department.”  (Taleb,  2007:184).  The  truth  is,  systems 
engineers  do  not  primarily  deal  with  questions  of  physics,  and  systems  engineering 
(particularly  in  the  DoD  and  NASA)  is  not  strictly  an  engineering  problem.  A  systems 
engineer’s  tool  kit  should  include  general  guidelines  and  principles,  and  not  solely  rely  on 
formal  rules  and  equations. 

Disciplines  like  systems  engineering,  systems  architecture  and  program 
management,  in  which  practitioners  confront  complex  problems  and  unique  situations, 
often  rely  on  heuristics,  which  Maier  and  Rechtin  describe  as  “trusted,  non-analytic 
guidelines  for  treating  complex,  inherently  unbounded,  ill-structured  problems.”  (Maier 
&  Rechtin,  2000:26)  Accordingly,  the  conclusions  of  this  research  are  summarized,  not  in 
a  statistical  table,  but  as  a  collection  of  heuristics.  These  rules  of  thumb  are  intended  to 
illuminate  facets  of  the  systems  engineering  art,  to  guide  system  developers,  to  encourage 
“reflective  practice,”  and  to  offer  memorable  summaries  of  reliable  principles. 
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Heuristics  are  general  principles,  not  hard-and-fast  rules.  They  should  be 
understood  as  vectors,  not  boundaries,  and  treated  as  suggestions,  not  commands.  Like 
zen  koans,  they  are  for  pondering,  for  thoughtful  consideration  and  experimentation.  It  is 
important  to  bear  in  mind  that  the  heuristics  listed  below  are  “not  proven  by  the  stories, 
but  are  designed  to  be  general  truths  that  are  illustrated  by  the  stories.”  (Laufer,  2000:xx) 
The  objective  of  this  list  is  to  expand  what  Schon  calls  the  practitioner’s 
“ repertoire  of  examples,  images,  understandings  and  actions.”  (Schon,  1983:138, 
emphasis  in  original).  Under  Schon’s  methodology,  practitioners  use  their  repertoire  to 
understand  unfamiliar  problems  in  tenns  of  familiar  ones,  and  unsolved  problems  in 
terms  of  solved  ones.  Expanding  the  repertoire  puts  more  tools  at  the  practitioner’s 
disposal,  enabling  the  community  so  understand  and  solve  problems  which  were 
previously  unfamiliar  or  unsolved. 

Table  4:  FIST  Heuristics 

1 .  Spending  less  time  gives  you  more  time. 

2.  To  finish  early,  start  early. 

3.  The  tortoise  was  faster  than  the  hare. 

4.  The  distance  between  planning  and  execution  should  be  as  short  as  possible  (i.e.  if 
you  wait  to  the  last  minute,  it  only  takes  a  minute  -  so  take  the  minute  now.) 

5.  If  I  don’t  have  enough  money,  don’t  give  me  more  time. 

6.  The  best  way  to  run  a  program  is  quickly.  (Gregory,  1985:162) 

7.  The  best  way  to  unleash  talent  is  not  to  have  too  much  of  it. 

8.  Talent  trumps  process. 

9.  Generalize  the  people,  specialize  the  tools  (the  Batman  Principle). 

10.  You  can’t  make  a  discovery  according  to  a  schedule. 

1 1 .  Don’t  deal  with  complexity  by  adding  more  complexity.  Deal  with  complexity  by 
removing  it. 

12.  Worse  is  better  (aka  the  best  is  the  enemy  of  the  good  enough). 

13.  Theory  Y  management  is  simpler  than  Theory  X. 
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14.  Complexity  and  reliability  are  inversely  proportional. 

15.  Only  ask  for  one  miracle  per  program.  (Rep.  Heather  Wilson,  House  Intelligence 
Committee) 

16.  Don’t  tinker  -  it  increases  complexity,  costs  time,  costs  money,  introduces 
instability. 

17.  Increasing  complexity  is  a  cost.  (Spinney,  1993:3) 

18.  Better,  faster,  cheaper  -  if  you  pick  two,  you’ll  only  get  two. 

19.  Better,  faster,  cheaper  -  pick  three. 

20.  The  project  leader’s  influence  over  the  development  is  inversely  proportional  to 
the  budget  and  schedule. 

2 1 .  FIST  failures  are  optimized  failures. 

One  item  on  the  list  is  of  particular  personal  interest  and  worth  a  brief  sidebar.  In 
1999,  the  apparent  failure  of  NASA’s  Faster,  Better,  Cheaper  initiative  helped  popularize 
the  heuristic  “faster,  better,  cheaper,  pick  two .”  One  of  this  research  project’s  objectives 
was  to  examine  this  heuristic  and  see  to  what  extent  it  is  a  reliable  guideline.  The  data 
indicates  the  “pick  two”  approach  is  unnecessary  at  best  and  a  counter-productive  self- 
fulfilling  prophecy  at  worst.  Several  of  the  referenced  projects  demonstrated  the 
possibility  of  delivering  a  project  that  is  simultaneously  fast,  inexpensive,  simple  and  tiny, 
while  still  providing  end  users  with  top-shelf  perfonnance  and  reliability. 

When  a  project  leader  decides  to  make  tradeoffs  between  cost,  schedule  and 
perfonnance,  focusing  on  two  while  sacrificing  the  third,  demonstrable  improvements  are 
often  made  in  two  areas,  to  the  detriment  of  the  third.  However,  it  need  not  be  this  way. 
When  project  leaders  refuse  to  sacrifice  any  of  the  three,  and  view  cost,  schedule  and 
perfonnance  as  not  only  interrelated  but  also  internally  reinforcing,  improvements  in  all 
three  areas  can  be  achieved  simultaneously.  Project  leaders  can  safely  reject  the  “pick 
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two”  heuristic,  on  the  grounds  that  it  needlessly  degrades  the  program  manager’s 
performance  as  well  as  the  perfonnance  of  the  developed  system. 

Recommendations  For  Further  Research 

This  thesis  explored  the  relationship  between  organizational  values  and 

operational  perfonnance  of  system  development  projects,  primarily  focusing  on 
spacecraft  and  military  aircraft.  These  two  technology  categories  were  selected  for 
several  reasons  -  they  are  high-profde,  well  documented,  and  often  troubled.  They  are 
also  strikingly  expensive,  and  thus  offer  an  opportunity  to  save  a  significant  amount  of 
money  by  improving  the  outcomes. 

Each  story  highlighted  a  different  aspect  of  the  relationship  between  the  FIST 
values  and  operational  outcomes.  Future  researchers  may  want  to  apply  the  FIST  rubric 
to  system  development  projects  I  did  not  examine,  such  as  the  Navy’s  F-14  and  F-18 
fighter  jets,  or  the  Air  Force’s  B-l,  B-2  and  B-52. 

Future  researchers  could  also  expand  the  scope  of  technology  under  consideration 
and  focus  on  software  projects  or  IT-related  efforts.  Software  development  projects  are 
increasingly  important  and  expensive.  Like  military  aircraft,  software  is  often  delivered 
late  and  over  budget.  Even  successfully  delivered  software  is  often  not  well  received  by 
users,  and  software  project  leaders  must  wrestle  with  issues  of  size  and  complexity.  It 
would  be  interesting  to  examine  whether  the  FIST  value  set  might  contribute  to  better 
outcomes  on  software  development  projects. 

The  thesis  focused  exclusively  on  American  projects.  Future  researchers  may 
want  to  look  at  the  international  community  and  examine  the  types  of  values  that  guide 
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development  of  military  aircraft  (or  other  types  of  systems)  in  countries  like  Sweden, 
France,  China  or  Russia. 

This  project  pointed  out  that  UAV’s  are  inherently  FISTy.  That  is,  the  typical 
measures  of  merit  for  a  UAV  are  whether  or  not  it  is  developed  quickly,  at  a  low  cost, 
and  whether  it  is  both  technically  and  operationally  simple.  Even  the  small  size  of  most 
UAV’s  is  a  point  of  pride.  It  would  be  interesting  to  examine  the  relationship  between  a 
genre’s  inherent  qualities  and  the  design  values  present  in  that  community.  Is  the  FIST 
value  set  really  embraced  by  UAV  developers,  or  is  it  simply  an  artifact  of  the 
technology’s  inherent  attributes?  If  such  a  distinction  exists,  does  it  matter?  Inversely, 
aircraft  carriers  seem  to  be  inherently  big,  expensive  and  complex.  A  future  researcher 
may  want  to  look  at  the  question  of  what  the  FIST  values  would  mean  in  the  context  of  a 
large  naval  ship  development  project. 

Finally,  the  FIST  approach  is  built  on  the  assumption  that  mature  technologies  are 
available  for  project  leaders  to  draw  from  as  they  develop  systems.  This  appears  to  rule 
out  using  the  FIST  approach  on  basic  research  projects  and  similar  scientific  endeavors, 
particularly  as  Simplicity  is  defined  and  understood,  but  perhaps  simplicity  might  be 
expressed  in  other  ways  in  the  context  of  basic  research.  Future  investigators  may  want  to 
examine  this  field  more  closely,  to  determine  whether  the  FIST  values  set  might  be  able 
to  contribute  to  the  “operational  success”  of  scientists  and  engineers  working  in  the 
earlier  phases  of  project  research  and  development. 
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Appendix  A:  FIST  Rubric 


FIST  Rubric 

High 

Medium 

Low/No 

Opposite 

10  pts 

5  pts 

0  pts 

-5  pts 

Fast  Is 

Strong,  explicit 

Modest,  occasional 

Little  to  no  mention 

Ambivalence  or 

Good 

affirmation  of  the 

affirmation  of  the 

of  the  importance  of 

antipathy  towards 

importance  of  speed. 

importance  of  speed, 
with  caveats. 

speed. 

speed. 

Formal  commitment  to 

No  commitment 

Explicit  support  for 

maintaining  deadline 

Modest,  informal 

(beyond  the  ordinary) 

"taking  as  much  time 

(contractual,  etc). 

commitment  to 
maintain  or  reduce 

to  maintain  deadline. 

as  we  need" 

Contractual  incentives 

deadline. 

No  steps  taken  to 

Active  attempts  to 

to  reward  early 

reduce  timeline  or 

increase  timeline. 

delivery. 

Contractual 

reward  on-time  / 

incentives  to  reward 

early  delivery. 

Accepts  schedule 

Concrete  steps  taken  to 

on-time  delivery. 

delays  rather  than 

actually  reduce 
development  timeline. 

Few  steps  taken  to 
reduce  timeline. 

accept  risks. 

Accepts  significant 
risks  in  order  to 

Accepts  moderate 

maintain  schedule. 

risks  in  order  to 
maintain  schedule 

Inexpensive 

Strong,  explicit 

Modest,  occasional 

Little  to  no  mention 

Ambivalence  or 

Is  Good 

affirmation  of  the 

affirmation  of  the 

of  the  importance  of 

antipathy  towards 

importance  of  low-cost. 

importance  of  low- 
cost,  with  caveats. 

low  cost. 

low-cost. 

Formal  commitment  to 

No  commitment 

Explicit  support  for 

maintain  budget 

Modest,  informal 

(beyond  the  ordinary) 

"spending  as  much 

(contractual,  etc) 

commitment  to 

to  maintain  budget. 

money  as  we  need 

maintain  or  reduce 

to" 

Contractual  incentives 

budget. 

No  steps  taken  to 

to  reward  cost  under- 

reduce  cost. 

Active  attempts  to 

runs. 

Few  concrete  steps 
taken  to  reduce  cost. 

increase  budget. 

Concrete  steps  taken  to 

Accepts  cost 

actually  reduce 

Accepts  moderate 

increases  rather  than 

development  cost. 

risks  in  order  to 
maintain  costs. 

accept  risks. 

Accepts  significant 
risks  in  order  to  reduce 

costs. 

Simple  Is 

Strong,  explicit  and 

Modest,  occasional 

Little  to  no  mention 

Ambivalence  or 

Good 

frequent  affirmation  of 

affirmation  of  the 

of  the  importance  of 

antipathy  towards 

the  importance  of 

importance  of 

simplicity. 

simplicity. 

simplicity. 

simplicity,  with 
caveats. 

No  mention  of  steps 

Explicit  support  for  / 

Deliberate  steps  taken 

taken  to  reduce 

pride  in  complexity. 

to  actually  reduce 

Modest  attempts  to 

complexity. 

complexity  in  many 

reduce  complexity  in 

Active  attempts  to  | 
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areas  (organization,  some  areas, 

technology, 
communication,  etc).  Primarily  relies  on 
mature  technologies, 
Heavy  reliance  on  with  some  new 

existing,  mature,  proven  developments, 
technology  (TRL  7+). 

System  capability 

System  capability  less  moderately  less  than 
than  previous  systems,  previous  systems. 

Frequent  emphasis  on  Occasional  mention 
the  importance  of  and  of  the  importance  of 
reliance  on  talent.  and  reliance  on  talent. 


No  mention  of 
technology  maturity 
levels. 

No  mention  of  the 
importance  of  and 
reliance  on  talent. 

System  capabilities 
match  or  moderately 
exceed  previous 
systems. 


increase  complexity. 

Heavy  reliance  on 
new  developments 
and  technology 
breakthroughs 

System  capabilities 
significantly  exceed 
previous  systems. 

Explicit  reliance  on 
formal,  structured 
process,  control  and 
compliance. 


Tiny  Is 
Good 


Strong,  explicit 
affirmation  of  the 
importance  of  small. 

Formal  commitment  to 
maintain  or  reduce  size. 

Concrete  steps  taken  to 
actually  reduce  size  (of 
org,  process,  sys, 
documentation,  etc). 


Modest,  occasional 
affirmation  of  the 
importance  of  small, 
with  caveats. 

Modest,  informal 
commitment  to 
maintain  or  reduce 


Few  steps  taken  to 
reduce  size. 


Little  to  no  mention 
of  the  importance  of 
small. 

No  commitment 
(beyond  the  ordinary) 
to  maintain  size. 

No  steps  taken  to 
reduce  size. 


Ambivalence  or 
antipathy  towards 
small,  lean  or 
streamlined 
approaches. 

Explicit  support  for  / 
pride  in  bigness. 

Active  attempts  to 
increase  size. 


Outcome 


Met  or  surpassed  all  or  most  operational 
requirements,  including  maintainability 
and  reliability 

Delivered  operational  capability 

Users  expressed  satisfaction 

Delivered  within  a  reasonable  margin  of 
original  cost  and  schedule  baseline 

Program  replicated  or  imitated  by 
subsequent  projects 


Mission  failed  to  meet  or  surpass  a  significant 
number  of  requirements 

System  rejected  by  users 

Program  cancelled  before  delivery 

Delivered  after  adding  substantial  funding  and 
substantial  schedule  increase 

Operational  use  is  severely  restricted  to  a  subset 
of  original  operational  vision  or  requirement  set. 
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Appendix  B:  Instructions  To  Co-Investigators 

This  project  explores  the  relationship  between  the  organizational  values  held  by 
technology  development  teams  and  the  operational  outcomes  of  the  systems  those  teams 
and  organizations  produce.  Specifically,  it  examines  the  impact  of  a  set  of  values  called 
FIST,  which  stands  for  Fast,  Inexpensive,  Simple  and  Tiny. 

Attached  to  these  instructions  you  will  find  a  brief  story  about  a  technology 
development  project  and  a  simple  coding  scheme  in  the  fonn  of  a  rubric.  A  rubric  is  a 
scoring  tool  for  making  subjective  assessments,  and  I  ask  that  you  use  the  rubric  to  guide 
your  assessment. 

It  is  important  to  take  a  moment  to  define  the  term  “values.”  A  value  is  a 
statement  of  priorities  and  preferences.  An  organization’s  values  typically  answer  the 
questions  “What  is  important?”  or  “What  is  good?”  When  an  organization  expresses  the 
FIST  values,  they  are  stating  “It  is  good  and  important  to  be  fast,  inexpensive,  simple  and 
tiny.” 

As  you  read  the  attached  story,  please  look  for  “value  clues”  which  indicate  the 
presence  of  the  FIST  values.  The  following  sentence  is  an  example  of  a  value  clue  related 
to  the  Inexpensive  value:  To  reduce  costs,  the  development  team  accepted  risks  that 
previous  teams  had  been  unwilling  to  endure.  This  statement  illustrates  that  cost 
reduction  was  more  important  than  risk  reduction,  specifically  as  compared  with  other, 
similar  teams.  Please  note  that  value  clues  will  not  always  be  explicit  and  obvious. 

The  project  need  not  meet  all  the  criteria  in  a  particular  category  in  order  to  merit 
a  particular  rating  (some  stories  may  not  mention  risk,  for  example),  but  it  should  meet 
enough  of  the  criteria  for  the  rating  to  be  reasonable  and  supportable. 

Along  with  the  organization’s  values,  you  will  also  be  asked  to  assess  the 
outcome  of  the  project.  Rather  than  focus  on  programmatic  success  (i.e.  on-time,  on- 
budget  delivery  of  a  system  that  satisfies  the  documented  requirements),  you  are  asked  to 
assess  the  operational  outcome  using  the  criteria  in  the  Outcome  portion  of  the  rubric. 

Thank  you  again  for  your  help  and  insight.  Please  feel  free  to  let  me  know  if  you 
have  any  questions. 

SYSTEM  NAME: 

F-Score: 

I-Score: 

S-Score: 

T-Score: 

Outcome: 
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Appendix  C:  Story  Data 


A  Note  About  The  Data 

The  following  pages  contain  excerpts  from  a  variety  of  reports,  newspaper  and 
magazine  articles,  books,  letters  and  interviews.  These  quotations  are  value  clues, 
selected  for  analysis  because  they  provide  some  insight  into  whether  the  project  leader’s 
organizational  and  design  values  align  with  the  FIST  value  set. 

The  excerpts  are  organized  under  five  headings  (Fast,  Inexpensive,  Simple,  Tiny 
and  Outcome),  depending  on  their  content.  In  many  cases,  a  particular  quotation  contains 
value  clues  related  to  two  or  more  of  the  FIST  elements.  In  those  cases,  in  order  to 
maintain  readability  and  context,  I  simply  included  it  once,  under  whichever  heading 
seemed  the  most  appropriate,  rather  than  post  the  same  quote  in  multiple  locations  or 
divide  a  single  quote  into  smaller  pieces.  Therefore,  a  project’s  lack  of  quotations  under 
any  given  heading  does  not  necessarily  reflect  an  absence  of  value  clues  related  to  that 
value. 

It  is  also  worth  noting  that  a  value  clue  may  indicate  either  the  presence  or  the 
absence  of  one  or  more  of  the  FIST  values.  A  project  may,  for  example,  have  a  large 
number  of  quotations  under  the  Fast  heading  and  a  very  low  F-score,  because  those 
quotations  indicate  a  rejection  or  absence  of  that  particular  value. 

After  collecting  and  analyzing  the  excerpts,  I  assigned  scores  and  grades  to  each 
project,  using  the  FIST  rubric  in  Appendix  A.  Because  many  statements  refer  to  multiple 
values,  several  projects  have  few  or  no  quotations  under  the  heading  Tiny  (for  example), 
while  still  earning  a  high  T-score.  This  is  simply  an  artifact  of  the  approach  to  grouping 
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quotations.  This  might  also  indicate  the  Tiny  value  was  an  implicit  rather  than  explicit 
value.  When  this  is  the  case,  it  is  discussed  in  the  analysis  section  for  that  project. 

Inputs  provided  by  the  five  co-evaluators  are  included  as  they  were  provided,  and 
were  not  edited  (other  than  the  occasional  spelling  correction).  No  attempt  was  made  to 
standardize  the  fonnat  of  these  inputs. 

Finally,  readers  should  understand  that  the  scores  assigned  to  the  projects  are 
necessarily  subjective,  and  were  deliberately  assigned  at  a  high  level  of  granularity,  to 
avoid  the  misleading  appearance  of  precision  in  what  was  emphatically  a  subjective 
analysis.  No  significance  is  attributed  to  the  relative  scores  within  each  category  -  a  high 
high  score  and  a  lower  high  score  are  treated  equally. 

The  conclusions  drawn  from  this  research  solely  depend  on  the  category  each 
system  ended  up  in  (High  or  Low),  not  on  the  specific  numerical  score  assigned  to  the 
system.  What  matters  was  simply  which  side  of  the  line  the  system  was  on,  not  how  far 
from  the  boundary  line  it  was.  Therefore,  the  conclusions  and  heuristics  would  not  be 
significantly  affected  if  a  project’s  score  was  changed  from  25  to  40  or  -20  to  0,  and  the 
reader  is  cautioned  to  not  be  distracted  by  the  numerical  scores  nor  to  draw  conclusions 
based  on  a  point  difference  of  10  or  15  points.  Similarly,  the  grades  were  assigned  using  a 
binary  pass/fail  (A  or  F)  scoring,  as  discussed  previously. 
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PROJECT  NAME:  NASA’s  Mars  Pathfinder 
DATES:  1993  -  1997 

STORY  SOURCE:  McCurdy,  2001;  Golombek,  1999;  McCurdy,  2005 
RUBRIC  SCORE:  40 
F -Score. TO 
I-Score:  10 
S-Score: 10 
T -Score: 10 
OUTCOME:  A 

SAMPLE  STATEMENTS: 

FAST 

“The  Pathfinder  team,  moreover,  was  asked  to  design,  build  and  prepare  their  spacecraft 
for  launch  in  just  3  years.  The  Viking  team  had  taken  6.”  (McCurdy,  2001:61) 

“Within  aerospace  circles,  the  cost  and  schedule  goals  given  to  the  Pathfinder  team  were 
widely  thought  to  be  impossible.”  (McCurdy,  2001:61) 

“To  leave  time  for  testing,  Spear  insisted  that  contractors  and  subsystem  managers 
deliver  their  hardware  no  later  than  halfway  through  the  development  cycle.  This  was  a 
demanding  requirement  inasmuch  as  the  development  phase  had  already  been  cut  to  just 
three  years.”  (McCurdy,  2001:130) 


INEXPENSIVE 

“Proponents  of  the  ‘faster,  better,  cheaper’  initiative  asked  the  Pathfinder  team  to  put  a 
lander  and  a  rover  on  the  surface  of  Mars  for  one-fourteenth  of  the  inflation-adjusted  cost 
of  the  1976  Viking  mission.”  (McCurdy,  2001:61) 

“To  fly  frequently,  exploration  advocates  had  to  fly  cheaply.”  (McCurdy,  2001:62) 

“They  called  it  ‘Pathfinder’  because  the  project  blazed  the  trail  for  a  new  generation  of 
low-cost  spacecraft.”  (McCurdy,  2001:62) 

“To  further  reduce  mission  costs,  the  Pathfinder  team  accepted  risks  that  Viking  team 
members  had  been  unwilling  to  endure.”  (McCurdy,  2001:65) 

“The  Pathfinder  team  clearly  saved  money  by  reducing  capability.”  (McCurdy,  2001:69) 

“The  Viking  team  spent  $27.3  million  to  develop  the  cameras  for  their  landers, 
approximately  $100  million  in  inflation-adjusted  dollars.  The  Pathfinder  team  spent  just 
$7.4  million.”  (McCurdy,  2001:70) 
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“[Donna]  Shirley  promised  to  build  a  free-ranging  rover  and  stay  within  NASA’s  $25 
million  cap.  To  avoid  cost  overruns,  she  relied  heavily  on  existing  technology.” 
(McCurdy,  2001:73) 

“The  Mars  Pathfinder  team  also  saved  money  by  employing  fewer  people.”  (McCurdy, 
2001:74) 

“Ownership  of  the  cost  constraint  permeated  all  the  way  down  to  the  lowest  level  of  the 
organization. . .  Team  members  learned  that  they  could  not  solve  a  problem  by  pulling  in 
more  people  or  spending  more  money.”  (McCurdy,  2001:98) 


SIMPLE 

“Land  it  directly  on  the  surface  of  Mars,  without  orbiting  or  descent  engines. . .” 
(McCurdy,  2001:62) 

“Team  members  launched  one  lander,  with  one  radio...”  (McCurdy,  2001:66) 

“. . .  the  most  complicated  piece  of  equipment  on  the  Pathfinder  mission  was  the 
Sojourner  rover,  which  cost  only  $25  million.”  (McCurdy,  2001:68) 

“Advantages  in  technology  allowed  Pathfinder  and  Sojourner  managers  to  develop 
scientific  instruments  without  the  cost  overruns  Viking  team  members  incurred.” 
(McCurdy,  2001:68-69) 

“By  the  mid  1990’s,  CCD  detectors  were  a  well-established  technology...”  (McCurdy, 
2001:71) 

“The  Pathfinder  team. . .  [concentrated]  project  management  at  a  single  center:  the  Jet 
Propulsion  Laboratory.”  (McCurdy,  2001:75) 

“They  did  not  include  orbital  spacecraft  or  as  many  scientific  instruments  as  part  of  their 
project.  They  did  not  operate  the  Pathfinder  spacecraft  for  a  long  period  of  time.” 
(McCurdy,  2001:75) 


TINY 

“Keeping  spacecraft  weight  low  enough  to  pennit  launch  on  a  Delta  2  was  a  major 
mission  goal.”  (McCurdy,  2001:64) 

“The  rover  team  had  to  develop  a  small,  semi-intelligent  robot,  about  2  feet  long  and  a 
foot  high  that  weighed  no  more  than  22  pounds. . .  It  had  to  operate  on  the  modest  amount 
of  power  generated  by  a  1.9  square  foot  solar  panel.”  (McCurdy,  2001:73) 

“The  central  management  team  never  comprised  more  than  about  30  people. . .  The 
science  advisory  teams  were  kept  small.”  (McCurdy,  2001:75) 

“Members  of  the  Pathfinder  team  employed  far  fewer  people  for  much  less  time.” 
(McCurdy,  2001:77) 
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“The  notion  that  a  small  team  of  JPL  engineers  and  scientists  could  develop  a  spacecraft 
in  less  than  three  years  for  under  $200  million  and  land  it  on  the  surface  of  another  planet 
was  contrary  to  the  history  and  culture  of  that  Center.  Most  people  thought  it  could  not  be 
done.”  (McCurdy,  2005:18) 


OUTCOME 

“. . .  the  Pathfinder  team  developed  their  spacecraft  in  3  years,  about  half  the  time  that  the 
Viking  team  took.”  (McCurdy,  2001:75) 

“. . .  the  Pathfinder  team  completed  their  project  with  about  one-third  of  the  number  of 
people  working  about  one-half  of  the  time.”  (McCurdy,  2001:75) 

“Compared  to  the  Viking  mission,  the  Pathfinder  team  saved  $3.6  billion  in  inflation- 
adjusted  currency.”  (McCurdy,  2001:62) 

“. . .  the  [landing]  system  worked  perfectly,  except  for  a  minor  glitch  when  a  lander  petal 
got  tangled  in  the  deflated  airbag.”  (McCurdy,  2001:67) 

“The  lander  bounced  at  least  15  times  up  to  12  in  high  without  airbag  rupture, 
demonstrating  the  robustness  of  this  landing  system.”  (Golombek,  1999) 

“The  cameras  on  the  Viking  and  Mars  Pathfinder  landers  had  approximately  the  same 
resolution,  but  the  Pathfinder  images  were  much  crisper  and  they  cost  substantially  less.” 
(McCurdy,  2001:71) 

“Existing  technology  in  the  1990’s  helped  the  Sojourner  team  develop  the  free-ranging 
rover  for  $25  million  without  cost  overruns.”  (McCurdy,  2001:74) 

“Many  people  worried  that  the  Pathfinder  experience  was  a  ‘one  time  only’  event,  an 
experiment  that  could  not  be  repeated  when  scientists  and  engineers  tried  to  apply  low- 
cost,  low-weight  technologies  to  a  broad  spectrum  of  spacecraft.  In  fact,  the  next  mission 
[Global  Surveyor ]  replicated  the  Pathfinder  experience.”  (McCurdy,  2001:120) 

“The  mission  operated  on  Mars  for  3  months  and  returned  2.3  Gbits  of  new  data, 
including  over  16,500  lander  and  550  rover  images,  16  chemical  analyses  of  rocks  and 
soil,  and  8.5  million  individual  temperature,  pressure  and  wind  measurements...  The 
mission  captured  the  imagination  of  the  public. . .  A  total  of  about  566  million  internet 
‘hits’  were  registered  during  the  first  month  of  the  mission,  with  47  million  hits  on  July 
8th  alone,  making  the  Pathfinder  landing  by  far  the  largest  internet  event  in  history  at  the 
time.”  (Golombek,  1999) 


ANALYSIS 

As  with  all  of  NASA’s  Faster,  Better,  Cheaper  initiative,  Pathfinder  explicitly 
affirmed  the  Fast  and  Inexpensive  values.  The  program  leaders  consistently  and 
deliberately  stated  that  it  was  important  and  good  for  the  program  to  be  Fast  and 
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Inexpensive,  and  program  success  would  be  determined  in  part  by  whether  or  not  the  cost 
and  schedule  constraints  were  followed.  There  was  a  similarly  strong  emphasis  on 
simplicity  and  smallness,  both  technical  and  organizational.  The  outcome  was  not  only 
successful,  it  was  successfully  replicated  on  a  subsequent  mission. 


CO-EVALUATOR  ANALYSIS 
RUBRIC  SCORE:  35 
F- Sc  ore: 5 
I-Score. TO 
S-Score: 1 0 
T -Score: 10 
OUTCOME:  A 

FAST  -  Medium  (5) 

The  MPF  project  timeline  was  dictated  at  program  outset:  3  years  from  design  to  launch, 
compared  to  6  years  for  Viking.  This  timeline  is  fast  for  any  type  of  space  mission,  let 
alone  one  to  explore  another  planet. 

No  evidence  was  presented  that  contractual  incentives  were  made  to  deliver  on  time. 
However,  the  decision  to  keep  development  in-house  gave  the  management  team 
increased  control  over  the  delivery  timelines  and  schedule  milestones. 


INEXPENSIVE  -  High  (10) 

Everything  about  the  Pathfinder  mission  was  focused  on  lowering  cost.  Even  the  name 
“Pathfinder”  was  symbolic  of  the  first  of  a  new  generation  of  lower  cost  spacecraft. 

A  primary  program  goal  for  the  MPF  team  was  to  “put  a  lander  and  a  rover  on  the  surface 
of  Mars  for  one-fourteenth  of  the  inflation-adjusted  cost  of  the  1976  Viking  mission.” 
The  MPF  team  saved  costs  in  all  areas  (design,  technology,  launch,  and  operations)  to 
complete  its  mission  for  $265  million  dollars,  which  is  extremely  inexpensive  for  this 
type  of  mission.  Deliberate  design  and  risk  acceptance  decisions  were  made  to  help 
achieve  this  cost  goal. 
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The  Inexpensive  value  drove  secondary  effects  in  tenns  of  simplicity  and  size  of  the 
technology  and  the  Pathfinder  team.  Those  effects  are  detailed  below. 

SIMPLE  -  High  (10) 

The  concept  for  actually  landing  the  Pathfinder  on  Mars  is  the  paragon  of  simplicity. 
Previous  Mars  and  moon  landing  missions  used  complex  engines  to  control  the  descent 
after  several  orbits.  MPF  went  to  a  simple  design  of  an  inflatable  airbag  to  protect  the 
lander  after  a  direct  insertion  into  the  atmosphere.  Even  simple  geometry  was  used  to 
ensure  the  lander  would  emerge  from  the  airbags  right-side  up. 

Simplicity  demanded  that  the  team  be  willing  to  take  risks  that  are  not  common  on 
interplanetary  missions.  These  included  only  having  one  lander,  one  radio,  a  landing 
point  that  was  easy  to  access  put  potentially  dangerous,  and  the  aforementioned  the 
landing  system. 

Simplicity  seemed  to  be  a  secondary  value  on  this  program.  Inexpensive  was  the  primary 
driver,  and  simplicity  had  to  be  enforced  to  bring  the  program  in  on  budget.  Simplicity 
was  implemented,  but  forced  a  reduction  in  capability.  These  capability  cutbacks  were 
often  to  the  disappointment  of  the  users  (i.e.  the  science  community),  who  had  to 
sacrifice  the  number  and  types  of  instruments,  as  well  as  the  duration  of  the  mission. 
While  the  Pathfinder  development  team  was  able  to  take  advantage  of  technical  learning 
curves  and  didn’t  have  to  pay  the  cost  of  invention  by  their  focus  on  simplicity,  the  users 
weren’t  getting  the  latest  cutting  edge  science  results.  According  to  the  FIST  principles, 
however,  reductions  in  capability  along  with  the  use  of  mature  technology  are  both 
expected  in  the  pursuit  of  simplicity, 

TINY  -  High  (10) 

Pathfinder  was  a  small  spacecraft,  deliberately  made  so  in  order  to  fit  on  the  most 
inexpensive  launch  vehicle  possible  (Delta  II). 

The  Pathfinder  development  and  operations  teams  were  deliberately  kept  small,  with 
most  of  the  development  work  done  in-house  at  JPL.  This  allowed  the  team  to  avoid 
hiring  people  to  coordinate  the  dispersed  activities  and  further  reduced  the  size  of  the 
management  and  advisory  teams.  The  intentionally  short  duration  and  simple  design  of 
the  mission  allowed  the  operations  team  to  be  kept  to  a  few  dozen  people.  Ultimately, 
the  value  of  “tiny”  kept  costs  down  as  well,  as  labor  expenses  are  a  major  portion  of  any 
technology  program’s  budget. 

Outcome  -  A 

The  Pathfinder  program’s  embodiment  of  the  FIST  principles  resulted  in  a  successful 
mission,  according  to  the  requirements  and  expectations.  The  team  met  its  budget, 
timeline,  and  mission  parameters.  It  took  risks  that  didn’t  result  in  adverse  outcomes  due 
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to  the  focus,  passion,  and  sheer  capability  of  the  people  on  the  team.  While  the 
Pathfinder  sacrificed  some  capabilities  in  tenns  of  redundancy,  scientific  innovation  and 
mission  duration,  these  issues  were  accepted  from  the  outset  given  the  “FIST-iness”  of 
the  program.  Overall  the  Pathfinder  program  was  judged  by  the  mission  team  to  be  a 
success  at  the  time  of  the  mission,  and  is  regarded  by  history  as  one  of  the  most 
successful  space  exploration  missions  ever. 
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PROJECT  NAME:  F-16  Falcon 
DATES:  1971  -  1979 

STORY  SOURCE:  Hehs,  1991;  USAF  F-16  Fact  Sheet;  Burton,  1993;  Air  Force 
Studies  Board,  2008 

RUBRIC  SCORE:  40 

F -Score:  10 
I-Score:  10 
S-Score:  10 
T -Score:  10 

OUTCOME:  A 
SAMPLE  STATEMENTS: 

FAST 

1971 :  Lightweight  fighter  RFP  sent  out 
December  1976:  F-16A  first  flight 

January  1979:  First  operational  F-16A  delivered  (USAF  F-16  Fact  Sheet) 

“We  were  ready  to  fly  the  lightweight  prototype  on  1  Feb  1974.  We  found  out  Northrop 
wasn’t  flying  until  June  or  July. . .  One  of  the  reasons  the  Air  Force  eventually  chose  our 
design  was  that  it  was  closer  to  a  full-scale  development  than  Northrop.”  (Hehs,  1991 :4) 

“The  development  timeline,  therefore,  was  shorter  for  the  individual  blocks  and  within  a 
management  and  oversight  time  frame.  This  approach  reduced  the  threat  of  instability,  in 
requirements  and  cost  because  the  expectations  were  not  only  measurable  but  near  term.” 
(AF  Studies  Board,  2008:32) 


INEXPENSIVE 

Unit  Cost:  $  14.6m  (FY98  dollars)  (USAF  F-16  Fact  Sheet) 

“If  I  plot  a  curve  of  cost  per  pound  for  succeeding  aircraft,  the  F-16  is  right  on  the 
curve. . .  However,  if  I  plot  a  curve  of  unit  flyaway  cost,  the  F-16  falls  off  that  curve.  It 
reversed  the  upward  trend  in  flyaway  cost.”  (Hehs,  1991) 

“ Most  importantly,  the  F-16  was  the  first  and  last  fighter  that  cost  less  than  its 
predecessor .”  (Burton,  1993:20;  emphasis  in  original) 


SIMPLE 

“We  were  perceived  as  being  anti-technology.  Our  slogan  was  ‘make  it  simple.’”  (Hehs, 
1991) 
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“We  used  the  technology  available  to  drive  the  given  end,  that  is,  or  was,  to  keep  things 
as  simple  and  small  as  we  could.  Our  design  was  a  finesse  approach.  If  we  wanted  to  fly 
faster,  we  made  the  drag  lower  by  reducing  size  and  adjusting  the  configuration  itself.  If 
we  wanted  greater  range,  we  made  the  plane  more  efficient,  more  compact.”  (Hehs,  1991) 

“Question:  What  were  some  of  the  conventions  the  fighter  mafia  challenged?  Answer: 
Range  was  associated  with  fuel  capacity.  High  speed  was  associated  with  bigger  engines. 
Technology  was  associated  with  complexity.  Twin  engine  designs  were  considered  safer. 
Size  and  cost  were  associated  with  capability.”  (Hehs,  1991) 

“We  were  well  aware  that  the  avionics  folks  would  be  putting  a  bunch  of  gadgets  into 
the  airplane,  which  would  increase  weight  and  decrease  perfonnance.  We  stacked  the 
deck.  We  made  the  airplane  so  dense  that  there  wasn’t  room  for  all  that  stuff.”  (Hehs, 

1991) 

“...  we  had  to  use  a  given  engine,  the  FI 00,  which  had  been  developed  for  the  F- 15...” 
(Hehs,  1991) 

“Question:  What  was  the  riskiest  portion  of  your  lightweight  fighter  design?  Answer:  The 
fly-by-wire  system.  If  the  fly  by  wire  didn’t  work,  our  relaxed  static  stability  wasn’t 
going  to  work. . .  We  had  a  backup  that  not  too  many  people  know  about.  We  designed 
the  fuselage  so  that  if  the  fly  by  wire  did  not  work,  we  could  go  back  to  a  statically  stable 
design  by  moving  the  wing  back.”  (Hehs,  1991) 

“In  designing  the  F-16,  advanced  aerospace  science  and  proven  reliable  systems  from 
other  aircraft  such  as  the  F-15  and  F-l  1 1  were  selected.  These  were  combined  to  simplify 
the  airplane  and  reduce  its  size,  purchase  price,  maintenance  costs  and  weight.”  (USAF 
F-16  Fact  Sheet) 

“...the  F-16  does  not  carry  the  expensive,  relatively  unreliable,  and  nonlethal  radar- 
guided  missile  that  seldom  had  been  effectively  used  at  long  range,  as  its  advocates 
promised,  prior  to  the  Gulf  War.”  (Burton,  1993:21) 

“The  YF-16  was  to  be  powered  by  a  single  Pratt  &  Whitney  FI 00  turbofan,  whereas  the 
YF-17  was  to  be  powered  by  a  pair  of  General  Electric  YJ101  engines.”  (Dewitte  an 
Vanhastel,  2008) 

“. .  .the  F-16  was  the  product  of  a  highly  disciplined  design  process.  The  combat  tasks 
were  decided  firt  and  were  not  allowed  to  vary  during  the  design  phase.  This  produced  a 
superior  design  that  was  tailored  to  those  specific  tasks.”  (Burton,  1993:133) 

“The  F-16  featured  many  innovations  in  the  application  of  engineering  and  management 
concepts,  but  fundamentally  the  advances  that  this  aircraft  possesses  reflect  the  shrewd 
application  of  available  technology.”  (AF  Studies  Board,  2008:32) 

“Follow-on  blocks  would  improve  the  combat  capability  in  a  predictable  and  stable 
engineering  and  management  environment.”  (AF  Studies  Board,  2008:32) 
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TINY 


“Another  reason,  besides  weight,  favors  small  size.  Smaller  aircraft  have  less  drag.  The 
airplane’s  exceptional  maneuverability  is  a  consequence  of  that  lower  drag  and  a  higher 
thrust-to-weight  ratio.”  (Hehs,  1991) 

“You  can,  for  example,  get  a  higher  thrust-to-weight  ratio  by  increasing  the  thrust.  You 
can  also  get  a  higher  thrust-to-weight  ratio  by  leaving  the  thrust  alone  and  reducing  the 
weight,  which  is  what  we  did  on  the  lightweight  fighter.”  (Hehs,  1991) 

“People  think  that  big  is  better.  It’s  not.  With  the  lightweight  fighter,  we  wanted  to 
achieve  our  ends  through  different  means.  We  increased  range  by  reducing  size.”  (Hehs, 
1991) 

“Question:  What  is  the  advantage  of  being  simple  and  small?  Answer:  In  general  terms,  it 
translates  into  lower  weight,  less  drag,  and  therefore  higher  performance.  Also  a 
fundamental  indicator  of  an  airplane’s  cost  is  its  weight.”  (Hehs,  1991) 

“So  the  engine  was  fixed.  That  meant  the  thrust  was  fixed.  If  we  wanted  a  high  thrust-to- 
weight  ratio,  we  had  no  choice  but  to  reduce  weight.”  (Hehs,  1991) 

“The  way  we  got  the  cost  down  was  by  getting  the  size  down.  That  was  another 
motivation  for  reducing  size.”  (Hehs,  1991) 

“The  Statement  of  Work...  reflected  the  Fighter  Mafia’s  philosophy — short  and  simple.  It 
was  only25  pages  long,  and  contractor  proposals  were  forbidden  to  be  longer  than  50 
pages,  a  revolution.”  (Burton,  1993:19) 


OUTCOME 

“In  an  air  combat  role,  the  F-16's  maneuverability  and  combat  radius  (distance  it  can  fly 
to  enter  air  combat,  stay,  fight  and  return)  exceed  that  of  all  potential  threat  fighter 
aircraft.”  (USAF  F-16  Fact  Sheet) 

“With  a  full  load  of  internal  fuel,  the  F-16  can  withstand  up  to  nine  G’s. . .  which  exceeds 
the  capability  of  other  current  fighter  aircraft.”  (USAF  F-16  Fact  Sheet) 

“combat  record  of  72  victories  with  0  losses”  (Lockheed-Martin  F-16  Product  Brochure) 

“Both  the  F-15  and  the  F-16  are  outstanding  fighters,  but  the  F-16  is  slightly  better. . .  [in 
Israel’s  1982  Middle  East  war]...  the  little  F-16  wound  up  shooting  down  more  enemy 
aircraft  than  its  big  brother.  It  also  suffered  fewer  hits  from  enemy  fire  and  had  less 
combat  damage  to  repair.”  (Burton,  1993:21) 


ANALYSIS 

The  F-16  development  team  explicitly  and  firmly  expressed  the  FIST  values. 
From  the  start,  they  affirmed  that  it  was  important  and  good  to  build  an  aircraft  that  was 
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inexpensive,  simple  and  tiny,  and  to  build  it  quickly.  The  project  leaders  made  constant 
design  decisions  to  ensure  the  aircraft  would  remain  low-cost,  small  and  simple,  and 
worked  hard  to  avoid  schedule  delays. 

While  the  Inexpensive,  Simple  and  Tiny  values  were  explicitly  held  by  the  F-16 
team,  Fast  was  an  implicit  value.  There  are  relatively  few  specific  statements  in  the  F-16 
literature  about  the  importance  of  a  short  development  timeline.  However,  it  was  clear  the 
project  leaders  recognized  the  importance  of  minimizing  the  development  timeline  and 
delivering  an  actual  aircraft  quickly.  They  did  this  informally,  without  an  externally- 
driven  deadline.  In  fact,  the  project  leaders  avoided  the  pitfall  of  speeding  by  “not 
working  against  an  arbitrary  schedule.”  (Hehs,  1991:4)  They  simply  worked  quickly, 
keeping  costs,  complexity  and  size  down,  and  delivered  sooner  than  their  competitor. 

Along  with  being  developed  in  half  the  time  and  at  half  the  cost  of  its  predecessor 
(the  F-15  Eagle),  the  F-16  has  an  impressive  operational  track  record.  The  Falcon  is  one 
of  the  most  successful  fighter  jets  ever  developed,  and  according  to  the  Lockheed  Martin 
F-16  Produce  Brochure,  “More  than  4,350  F-16s  have  been  produced  for  24  countries.”  It 
is  clearly  a  successful  project.  However,  the  development  effort  was  not  imitated  by 
subsequent  development  teams. 

Later  versions  of  the  F-16  were  not  developed  with  the  same  value  set,  and 
additional  costs,  delays  and  complexities  were  added  on,  blurring  the  idealistic  vision  of 
the  Falcon’s  originators.  As  Burton  describes  it,  “The  establishment  eventually  ruined  the 
later  versions  of  the  F-16  by  adding  tons  of  high-tech  gadgetry,  which  increased  its 
weight  and  cost  and  significantly  degraded  its  performance.”  (Burton,  1993:26).  None  the 
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less,  the  initial  delivery  provided  a  high-perfonnance,  FISTy  aircraft  which  has  continued 
to  perform  well  around  the  world,  despite  being  “ruined”  in  later  versions. 

CO-EVALUATOR  ANALYSIS 
RUBRIC  SCORE:  35 
F -Score:  10 
I-Score:  1 0 
S-Score:  1 0 
T -Score. TO 
OUTCOME:  A 

I  wanted  to  be  methodical  and  try  to  keep  as  much  of  my  own  bias  out  of  it.  I  basically 
higlighted  any  place  in  the  document  that  related  to  a  F.I.S.T.  value.  What  flushed  out,  to 
me,  was  a  natural  comparison  between  the  F-15  and  F-16.  There  were  obviously  many 
other  mentions  with  relation  to  other  stuff,  and  it  wasn’t  always  clear  how  I  should 
attribute  statements. 

So,  for  the  purposes  of  distillation,  I  took  anything  that  related  to  the  "Fighter  Mafia"  or 
austerity  advocates,  or  (later  in  the  document)  any  of  the  gov't  officials  who  were  pushing 
for  austerity  as  being  F-16. 
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PROJECT  NAME:  NASA  Near  Earth  Asteroid  Rendezvous  (NEAR) 

STORY  SOURCE:  Laufer,  2000;  McCurdy  2005 
DATES:  1993  -  1996 
RUBRIC  SCORE:  40 
F -Score:  10 
I-Score:  10 
S-Score:  10 
T -Score:  10 
OUTCOME:  A 

SAMPLE  STATEMENTS: 

FAST 

.  the  NEAR  spacecraft  would  have  to  be  launched  during  a  1 -minute  crack  in  a  2-day 
window  in  mid-February  1996.  If  we  missed  that  narrow  window,  NASA  would  have  to 
wait  7  years  for  another  window. . .”  (Laufer  2000: 122) 

“The  tight  window  turned  out  to  be  our  best  ally,  because  it  created  a  definite  sense  of 
urgency  for  everyone  involved  and  focused  all  attention  on  that  goal.  (Laufer  2000: 122) 

“. . .  we  had  to  hurry. . .  Every  Monday  morning  each  of  our  lead  engineers  would  give  a 
3-minute  report  on  how  they  were  progressing  against  the  simple  12-line  schedule  I  had 
developed. . .  we  would  finish  in  40  minutes  and  leave  the  meeting  armed  with  a  list  of 
concrete  action  items  that  could  be  followed  and  tracked.”  (Laufer  2000:122) 

“Developing  a  spacecraft  typically  takes  three  to  four  years.  Discovery  allowed  36 
months  for  development.  NEAR  was  delivered  in  just  27  months.”  (Laufer  2000:123) 

“[NEAR  program  manager  Thomas]  Coughlin  had  a  clear  vision  of  the  management 
philosophy  necessary  to  produce  a  low-cost,  short-schedule  spacecraft...”  (McCurdy, 
2005:19) 

“. . .  the  NEAR  team  had  given  themselves  considerably  less  time.”  (McCurdy  2005:17) 

“Had  the  NEAR  team  taken  on  too  much,  promising  to  build  and  test  a  spacecraft  too 
rapidly  with  too  little  money?  Team  leaders  were  confident  they  had  not.  They  did  not 
believe  that  they  had  taken  on  a  task  that  would  prove  too  fast  or  too  cheap.”  (McCurdy 
2005:18) 

“The  two-year  development  schedule  was  the  ‘foremost  design  driver’  for  the  spacecraft 
and  its  mechanisms,  team  members  noted.  Components  had  to  be  designed  and 
manufactured  in  a  remarkably  short  period  of  time.  Team  leaders  planned  to  begin 
assembling  the  whole  spacecraft  just  18  months  after  funding  began.”  (McCurdy  2006:20) 
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“Most  impressive  was  the  short  development  time.  An  analysis  of  NASA  space  programs 
suggested  that  planetary  missions  required  an  average  of  eight  years  of  preparation;  a 
subsequent  review  of  spacecraft  difficulties  identified  insufficient  development  time  as  a 
primary  cause  of  mission  failure.  The  NEAR  spacecraft  team  produced  a  simple 
spacecraft — from  funding  to  launch — in  just  27  months.”  (McCurdy  2005:20) 


INEXPENSIVE 

“A  second  great  motivator  was  the  chance  to  prove  to  NASA  that  the  NEAR  team  could 
deliver  the  program  at  a  much  lower  cost  than  expected.  NASA’s  first  projections 
budgeted  NEAR  at  $200  million  or  more,  but  we  estimated  the  project  at  $1 12.  NASA 
was  uneasy  with  our  bid  and  budgeted  NEAR  at  $150  million;  we  believed  we  could  do  it 
for  less.”  (Laufer  2000: 122) 

“. . .  the  NEAR  team  had  promised  to  finish  the  first  phase  of  the  project — construction  of 
the  spacecraft — for  a  paltry  $122  million,  a  considerable  commitment  to  the  low-cost 
philosophy.”  (McCurdy  2005:  17) 

“Based  on  the  funds  devoted  to  the  development  of  the  spacecraft,  NEAR  sat  near  the 
bottom  of  the  list.”  (McCurdy  2005: 18) 

“The  NEAR  team  spent  $79  million  designing,  constructing  and  testing  its  spacecraft, 
excluding  scientific  instruments...  These  were  small  sums  by  exploration  standards.” 
(McCurdy  2005:20) 


SIMPLE 

“Had  I  incorporated  even  half  of  these  good  ideas,  the  spacecraft  would  never  have  been 
built.  Only  those  changes  that  could  be  made  with  negligible  or  minimal  disruption  were 
even  considered.”  (Laufer  2000: 123) 

“By  pushing  specs,  cost,  and  trade-offs,  and  driving  decision  making  to  the  lowest  level 
possible,  we  were  able  to  focus  on  the  big  picture  as  laid  out  in  our  simple  schedule  rather 
than  the  engineering  problems  and  surprises  that  normally  plague  a  fast  project.”  (Laufer 
2000:123) 

“The  NEAR  and  Pathfinder  teams  sought  to  produce  spacecraft  that  were  simultaneously 
cheap  and  reliable  and  to  produce  them  rapidly.  The  NEAR  team  sought  to  achieve  this 
goal  through  simple  design,  high-tech  instrumentation,  reduced  launch  costs,  calculated 
risk-taking,  and  team-based  management  techniques.”  (McCurdy  2005:19) 

“To  reduce  the  possibility  of  interactive  flaws,  team  members  needed  a  relatively  simple 
spacecraft.  In  pursuit  of  mechanical  simplicity,  the  NEAR  team  designed  a  spacecraft 
with  few  moving  parts.”  (McCurdy  2005:  19) 

“The  simplified  design  reduced  cost,  cut  time,  and  helped  prevent  unexpected  integration 
problems  as  the  launch  date  approached.  It  also  created  a  spacecraft  that  was  a  bit  hard  to 
handle.”  (McCurdy  2005:20) 
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TINY 


OUTCOME 

“NEAR  chased  the  asteroid  Eros  around  the  solar  system  for  four  years,  finally  orbiting 
the  21 -mile-long  object  in  February  2000.  After  circling  Eros  for  one  year,  the  renamed 
NEAR-Shoemaker  spacecraft  touched  down  on  the  asteroid’s  surface  in  February  2001, 
the  first  such  landing  in  the  history  of  spaceflight.”  (McCurdy  2005:2) 

“Today,  at  3:02:10  EST,  NASA’s  NEAR  Shoemaker  spacecraft  traveled  its  last  mile, 
cruising  to  the  surface  of  asteroid  Eros  at  a  gentle  4  mph. . .  finally  coming  to  rest  after  its 
2-billion-mile  journey...  it  collected  10  times  more  data  than  originally  planned  and 
completed  all  its  science  goals  before  attempting  its  descent  to  the  asteroid. . .  When  it 
touched  down,  NEAR  shoemaker  became  the  first  spacecraft  ever  to  land,  or  even 
attempt  to  land  on  an  asteroid.  The  success  was  sweetened  by  the  fact  that  NEAR 
Shoemaker  was  not  designed  as  a  lander.”  (NASA  Press  Release,  12  Feb  2001) 


ANALYSIS 

This  project  involved  a  very  explicit  adoption  of  the  FIST  values.  The  project 
leaders  consistently  affirmed  that  it  was  important  and  good  to  be  Fast,  Inexpensive, 
Simple  and  Tiny,  both  in  terms  of  technology  and  organization. 

Engineers  gave  3-minute  reports  and  used  a  simple  12-line  schedule.  While  these 
relate  to  Fast  and  Simple,  they  also  are  evidence  of  an  organization  that  values  Tiny. 
Many  so-called  “good  ideas”  were  rejected  because  they  would  have  increased  the  cost, 
schedule  or  complexity  of  the  project,  demonstrating  that  project  leaders  were  unwilling 
to  sacrifice  cost,  schedule  or  complexity  in  order  to  incorporate  “high  tech”  subsystems 
and  components.  Decision-making  was  pushed  to  the  lowest  level,  simplifying  and 
miniaturizing  the  organization.  Even  ease  of  performance  was  sacrificed  in  the  name  of 
simplicity,  a  remarkably  bold  decision. 
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The  project  was  an  enormous  success,  providing  an  order  of  magnitude  more  data 
than  originally  expected.  It  even  landed  on  the  asteroid  Eros,  the  first  time  such  a 
maneuver  had  ever  been  attempted,  despite  the  fact  that  it  was  not  designed  to  be  a  lander. 


CO-EVALUATOR  ANALYSIS 

Fast:  10 

Similar  previous  systems  took  up  to  seven  years  to  develop,  NEAR  was  scheduled  to  be 
launched  just  27  months  after  funding. 

This  schedule  is  referred  to  as  aggressive  several  places  in  the  document. 

The  deadline  was  a  make  or  break  target,  if  they  missed  their  launch  window,  it  could 
complete  its  mission,  and  this  was  made  clear  to  the  team. 

Inexpensive:  10 

The  proposed  budget  was  markedly  below  similar  programs  and  almost  30%  below 
allowed  budget,  and  they  still  came  in  under  budget. 

The  team  was  so  proud  of  coming  in  under  budget  that  they  held  a  ceremony  (big  check 
and  all)  to  return  the  unused  portion  to  NASA. 

Took  risks  to  accomplish  this,  including  using  solar  cells  (cheaper)  in  an  environment 
that  they  were  traditionally  not  used  because  the  distance  from  the  sun  limits  their 
effectiveness. 

Simple:  10 

Traditional  probes  used  many  scientific  instruments,  which  drove  up  complexity.  But 
NEAR  committed  to  only  using  2  instruments. 

The  team  was  limited  to  only  on  mission  scientist  to  simplify  the  organizational 
relationships. 

Team  focused  on  using  high  TRL  and  very  few  moving  parts  (fixed  solar  cells).  To 
make  this  work,  the  team  had  to  use  inventive  ideas  in  the  arrangement  of  the 
components. 

Tiny:  10 

NEAR  was  limited  in  size  by  the  specific  launcher  it  was  authorized  to  fly  on. 

The  result  was  a  vehicle  that  was  almost  50%  less  mass  than  previous  probes. 
Additionally,  the  delta  v  budget  was  very  tight,  forcing  the  vehicle  to  stay  within  its  size 
budget  or  risk  going  over  total  weight  budget. 

The  team  was  very  aware  of  the  size  issues,  and  were  worried  that  the  vehicle  would  be 
over  budget  all  the  way  up  to  the  official  weighing  day  shortly  before  launch. 

Outcome:  A 

NEAR  was  the  first  landing  on  an  asteroid,  and  was  never  designed  to  do  so 
(demonstrates  both  flexibility  of  the  hardware  and  the  operations  team) 
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Quote  from  the  paper  -  "though  'faster,  cheaper,  better’  is  not  in  use,  new  programs 
contain  the  core  features"  and  it  is  implied  that  the  success  of  NEAR  played  a  role  in  this 
cultural  shift. 

The  researchers  were  very  happy  with  the  science  from  NEAR. 

It  is  interesting  to  note  that  several  programs  tried  to  replicate  the  processes  used  by 
NEAR.  Unfortunately,  most  cut  the  budgets  and  staff  too  close  to  the  bone  and  wound  up 
being  on  the  wrong  side  of  FIST  (too  fast,  too  inexpensive,  too  simple,  too  tiny),  leading 
to  an  official  movement  away  from  "faster,  cheaper,  better".  But,  over  time,  the  more 
balanced  approach  has  seeped  into  some  elements  of  the  culture  as  reported  above. 
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PROJECT  NAME:  Skunk  Works 
STORY  SOURCE:  Skunk  Works,  Rich,  1994 

DATES: 

RUBRIC  SCORE:  40 

F -Score:  10 
I-Score:  10 
S-Score:  10 
T -Score:  10 

OUTCOME:  A 

SAMPLE  STATEMENTS: 

FAST 

P-80:  “Kelly’s  guys  [built]...  the  cigar-shaped  prototype  of  theP-80  Shooting  Star,  in 
only  143  days — 37  days  ahead  of  schedule.”  (pp  1 12) 

HAVE  BLUE:  “We  built  two  Have  Blue  prototypes  in  record  time,  only  twenty  months 
from  the  day  the  contract  was  awarded  until  I  made  the  first  flight.”  (pp  57) 

F- 117:  “The  first  stealth  fighter  squadron,  composed  of  eighteen  airplanes  and  a  few 
spares,  was  ready  to  go  to  battle  only  five  years  after  the  initial  Air  Force  go-ahead. . .” 

(PP  91) 

F- 117:  “We  started  assembly  the  same  time  as  McDonnel  Douglas  started  the  F-18 
fighter.  They  took  ten  years  to  produce  their  first  operational  squadron  of  twenty 
airplanes.  We  took  only  five  years.  And  theirs  was  a  conventional  airplane,  while  ours 
was  entirely  revolutionary  technology.”  (pp  80) 

U-2:  “To  put  an  airplane  in  the  sky  in  only  eight  months  was  a  tremendous  achievement.” 
(PP  132) 

INEXPENSIVE 

“Lockheed’s  management  agreed  that  Kelly  could  keep  his  tiny  research  and 
development  operation  running — the  first  in  the  aviation  industry — as  long  as  it  was  kept 
on  a  shoestring  budget. . .  So  Kelly  and  a  handful  of  bright  young  designers  he  selected 
took  over  some  empty  space  in  Building  82. . .  That  way  the  overhead  was  kept  low  and 
the  financial  risks  to  the  company  stayed  small.”  (pp  112  -  113) 

HAVE  BLUE:  “It  was  built  on  the  cheap  all  the  way. . .”  (pp  58) 

F- 1 17:  “We  built  eight  a  year  at  a  fly-away  cost  of  $43  million  each.  Stealth  did  not  come 
cheap,  but  considering  the  revolutionary  nature  of  the  product  and  the  enormous  strategic 
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advantages  it  afforded,  the  F-l  17A  was  the  most  cost  effective  new  weapons  system  in 
the  inventory.”  (pp  95) 

F-l  17:  . .  we  achieved  phenomenal  efficiency. . .  At  one  point  I  offered  to  give  the 

government  some  of  its  money  back. . .  (pp  95) 

F-l  17:  “. . .  they  actually  developed  this  automated  program  in  only  120  days  and  at  a  cost 
of  only  $2.5  million.  It  was  so  advanced  over  any  other  program  that  the  Air  Force 
bought  it  for  use  in  all  their  attack  airplanes.”  (pp  95) 


SIMPLE 

“There  shall  be  only  one  object:  to  get  a  good  airplane  built  on  time. . .  Any  cause  for 
delay  shall  be  immediately  reported  to  C.L.  Johnson. . .  Everything  possible  will  be  done 
to  save  time.”  (pp  1 16) 

“The  reason  why  Kelly  could  move  so  quickly  building  the  U-2  was  that  he  could  use  the 
same  tools  from  the  prototype  of  the  XF-104  fighter. . .  Using  that  tooling  would  save 
many  months  and  a  lot  of  money. . .  Each  airplane  could  cost  the  American  taxpayers  $  1 
million,  including  all  development  costs,  making  it  the  greatest  procurement  bargain  ever.” 

(PP  131) 

TINY 

U-2:  “We  had  on  hand  only  four  engineers  and  twenty  maintenance,  supply,  and 
administration  people.  Nowadays  they’d  probably  use  twenty  people  just  to  fuel  an 
airplane.”  (pp  134) 


OUTCOME 

U-2:  “there  never  has  been  a  single  day  since  that  airplane  became  operational  in  1956 
that  a  U-2  isn’t  flying  somewhere  in  the  world  on  a  surveillance  operation  for  the  blue- 
suiters,  NASA  or  the  Drug  Enforcement  Agency.  In  fact,  on  more  than  one  occasion  over 
the  years,  the  U-2  may  have  saved  the  world  from  thennonuclear  war.”  (pp  178) 


ANALYSIS 

The  Lockheed  Skunk  Works  shop  is  the  prototypical  FIST  organization. 
Throughout  its  long  history,  it  has  consistently  and  firmly  asserted  that  it  is  good  and 
important  for  its  projects  to  be  Fast,  Inexpensive,  Simple  and  Tiny.  Projects  which  do  not 
meet  those  criteria  are  rejected.  It  shows  that  it  is  possible  for  an  organization  to 
consistently  implement  the  FIST  values  and  produce  effective  systems. 
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Skunkworks  reliably  delivered  simple,  low-cost,  high-tech  aircraft  on  short 
timelines,  to  support  the  DoD  and  CIA’s  highest  priority  missions.  These  systems  went 
on  to  be  used  effectively  across  a  wide  spectrum  of  operations.  The  Skunk  Works 
approach  is  widely  imitated,  from  Raytheon’s  Bike  Shop  to  the  Office  of  Naval 
Research’s  Swamp  Works  to  countless  other  lesser-known  “skunk  works”  units  in 
organizations  across  various  industries. 
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PROJECT  NAME:  PAVE  LOW  III 
DATES:  1975  -  1980 

STORY  SOURCE:  Gambone,  1988; 

RUBRIC  SCORE:  40 
F -Score: 10 
I-Score:  1 0 
S-Score:  1 0 
T -Score. TO 
OUTCOME:  A 

SAMPLE  STATEMENTS: 

FAST 

“The  schedule,  being  so  short  and  rigid,  required  self-motivated  personnel  with  high 
morale  and  personal  initiative.”  (Gambone,  1988:23) 


INEXPENSIVE 

“He  [Terry]  also  thought  it  could  be  done  for  $3  million.”  (Gambone,  1988:24) 

“...  the  ROC  [Required  Operational  Capability]  emphatically  indicated  that  cost  would 
be  a  determining  factor  in  acceptance  of  any  future  effort  on  this  ROC.”  (Gambone, 
1988:40) 

“Since  the  APQ  141  was  only  a  flight  test  radar  and  the  APQ  148  was  only  a  TA  radar, 
the  development  costs  and  program  risks  associated  with  these  radars  were  high.  Thus, 
these  systems  were  eliminated.”  (Gambone,  1988:40) 

“Since  all  systems  had  approximately  the  same  performance,  the  cost  and  interface 
problems  were  the  deciding  factors.  CMC  agreed  to  loan  PAVE  LOW  a  Doppler  nav 
system  free  of  cost.  In  addition  the  CMC  208V  required  no  interface  hardware,  thus  it 
was  chosen.”  (Gambone,  1988:41) 


SIMPLE 

“Some  people  believed  that  the  Gunship  team  was  the  most  talented  group  within  ASD  to 
accomplish  the  tasks  required.”  (Gambone,  1988:23) 

“He  [Terry]  discussed  the  situation  with  Colonel  Bell,  director  of  the  ASD  SPO  for  PL  II, 
and  suggested  that  the  program  could  be  done  in-house  utilizing  the  Gunship  personnel.” 
(Gambone,  1988:24) 
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“Since  the  computer  interfaces  with  many  of  the  avionics  systems  a  tried  and  proven 
computer  was  needed.  The  prime  candidate  was  the  IBM  CP993  computer.  This 
computer  was  picked  for  the  following  reasons:  a.)  Prototype  hardware  was  available  at 
WPAFB  for  no  additional  costs  to  the  Air  Force,  b.)  The  Program  Office  had  extensive 
experience  with  this  computer. . .  d.)  The  IBM  computer  was  in  the  Air  Force  inventory.” 
(Gambone,  1988:42) 


TINY 

“Colonel  Terry  had  a  staff  of  one  civilian,  one  military  officer,  and  a  secretary.  There  was 
a  test  organization  of  approximately  four  people  who  were  in  charge  of  flight  testing. 
There  was  also  a  large  procurement  section  and  a  large  engineering  section.  There  were 
no  ‘managers.’”  (Gambone,  1988:21) 

“The  preliminary  planning. . .  was  accomplished  over  several  months  using  a  small 
selected  group  of  personnel.”  (Gambone,  1988:24) 

“In  addition  to  cost,  other  factors  such  as  perfonnance,  size,  weight,  risk,  reliability  and 
inventory  status  were  used  for  selecting  equipment.”  (Gambone,  1988:40) 

“. . .  since  the  FLIR  would  be  mounted  on  the  nose  of  the  a/c  it  was  also  necessary  that  it 
be  compact  in  size  and  weight.”  (Gambone,  1988:40) 

“For  picking  a  TF/TA  radar  the  most  significant  factors  were  risk,  or  development  cost, 
and  the  size.”  (Gambone,  1988:40) 

“The  LN15S  was  picked  since  it  h  ad  less  weight,  cost,  size,  and  better  performance.” 
(Gambone,  1988:41) 


OUTCOME 

“In  1990,  Pave  Lows  from  the  20th  Special  Operations  Squadron  led  the  way  for  Anny 
AH-64  Apaches  during  an  air  strike,  thus  opening  the  air  war  in  Operation  Desert  Storm.” 
(Air  Force  MH-53J  Fact  Sheet) 

“MH-53J's  were  used  in  a  variety  of  missions  during  Desert  Stonn.  Pave  Lows  were 
among  the  first  aircraft  into  Iraq  when  they  led  Army  AH-64  Apaches  to  destroy  Iraqi 
early  warning  radars  and  opened  a  hole  in  enemy  air  defenses  for  the  opening  air  annada. 
In  addition  to  infiltration,  exfiltration  and  resupply  of  special  forces  teams  throughout 
Iraq  and  Kuwait,  Pave  Lows  provided  search  and  rescue  coverage  for  coalition  air  forces 
in  Iraq,  Saudi  Arabia,  Kuwait,  Turkey  and  the  Persian  Gulf.  An  MH-53J  made  the  first 
successful  combat  recovery  of  a  downed  pilot  in  Desert  Stonn.  Following  the  war,  MH- 
53 J’s  were  deployed  to  Northern  Iraq  to  support  Operation  Provide  Comfort,  assisting 
displaced  Kurds.  Pave  Lows  were  also  used  extensively  during  Operation  Just  Cause  in 
Panama.”  (GlobalSecurity.org  MH-53J  Pave  Low  III  page,  2006) 
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ANALYSIS 

Gambone’s  story  of  the  Pave  Low  Ill’s  history  is  full  of  value  clues,  explicitly 
stating  the  reasons  for  various  judgments  and  decisions  made  on  the  project.  These  value 
clues  inevitably  point  to  the  FIST  value  set  at  work  within  the  Pave  Low  III  project. 

When  faced  with  a  short,  rigid  schedule,  the  Gunship  SPO  responded  by  applying 
talented,  motivated  people,  rather  than  complaining  about  inadequate  time  or  resources 
(as  happened  in  the  FIA  project).  This  shows  a  strong  appreciation  for  the  importance  of 
rapid  development  and  delivery,  i.e.  the  Fast  value.  When  faced  with  the  opportunity  to 
use  immature  components  that  would  have  required  extensive  and  expensive  testing  (and 
thus  had  a  high  risk  of  failure  and  delay),  project  leaders  decided  to  simply  eliminate  the 
sub-system  rather  than  incur  additional  cost  and  schedule  risk.  This  willingness  to 
sacrifice  performance  and  functions  rather  than  increase  the  system’s  cost,  schedule  and 
complexity  is  clear  evidence  of  the  FIST  values  at  work.  Project  leaders  deemed  it  better 
to  keep  the  cost  and  complexity  down  than  to  add  an  expensive  subsystem  or  function. 

The  Pave  Low  III  program  is  one  of  the  most  explicit  examples  of  the  Tiny  value 
being  applied  to  both  the  organization  and  the  technology.  Gambone’s  story  frequently 
refers  to  the  team’s  preference  for  small  components  and  small  work  teams.  Rather  than 
rely  on  formal  processes  and  structures,  the  Gunship  SPO  explicitly  relied  on  a  small 
group  of  talented,  hand-picked  personnel  as  the  means  of  ensuring  a  positive  outcome. 
This  not  only  indicates  a  strong  commitment  to  the  Tiny  value,  but  also  a  deep 
understanding  of  the  importance  of  being  Simple  and  the  relationship  between  these  two 
values.  Recall  from  the  FIST  Rubric,  reliance  on  people  is  an  indication  of  Simplicity, 
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while  reliance  on  formal  structures  and  processes  tends  to  indicate  a  preference  for 
complexity. 

The  Pave  Low  III  has  an  impressive  operational  record,  following  the  example  of 
earlier  versions  of  the  Pave  Low  family.  It  was  one  of  the  first  aircraft  to  enter  Iraqi 
airspace  in  the  first  Gulf  War,  and  provided  a  wide  range  of  support  to  special  forces 
teams.  It  is  clearly  an  operational  success. 
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PROJECT  NAME:  P-51  Mustang 
DATES:  1939  -  1942 

STORY  SOURCE:  Nelson,  1944;  Carson,  1985;  Sanders,  1942;  Anon,  1975;  Bjamoe, 
2006 

RUBRIC  SCORE: 

F -Score:  10 
I-Score:  5 
S-Score:  10 
T- Score:  10 

OUTCOME:  A 
SAMPLE  STATEMENTS: 

FAST 

“The  original  Mustang,  designed  and  built  for  the  British  in  less  than  120  days. . 

(Nelson,  1944:127) 

“[North  American]  had  probably  been  working  on  drawings  of  what  was  to  become  the 
P-5 1  as  early  as  the  previous  summer,  well  before  the  official  go-ahead.  Consequently, 
when  historians  point  out  that  the  prototype  was  completed  in  only  117  days,  they 
conveniently  overlook  the  fact  that  actual  work  had  begun  at  least  three  and  possibly  four 
months  prior  to  the  May  1  official  start  date.”  (Carson,  1985:33) 

“. . .  they  already  had  a  clear  idea  of  what  they  wanted  and. . .  they  used  the  data  from 
Curtis. . .  to  cut  trial  and  error  time  and  to  find  tune  their  own  concept  of  what  a  modem 
fighter  should  be.”  (Carson,  1985:33) 

“. . .  North  American  promised  to  deliver  320  of  the  new  aircraft  before  September  1941, 
only  17  months  after  the  signing  of  the  agreement...”  (Carson,  1985:33) 

“It  must  have  been  obvious  to  the  British,  that  development  of  the  NA-7 1  [the  P-5 1  ’s 
original  designation]  was  well  along  when  they  signed  their  initial  contract  on  May  1, 
1940...”  (Carson,  1985:33) 

“. . .  when  developing  the  P-5 1  a  kind  of  modularization  and  parallelization  of  the  sub¬ 
processes  was  applied.  The  airframe  was  for  example  divided  into  sections  which  could 
be  developed  independently  as  the  interfaces  were  defined  from  the  beginning  which  is 
the  same  principle  used  in  contemporary  modular  product- architectures.  This  made  it 
possible  to  execute  parallel  product  development  and  set-based  design,  which  means  that 
several  designs  of  the  same  module  were  developed  concurrently  and  consequently  the 
decision  about  the  final  design  could  be  postponed.  The  result  of  this  made  it  possible  to 
work  with  new,  advanced  designs  in  a  very  limited  time  frame.”  (Bjarnoe,  2006:44) 

INEXPENSIVE 
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SIMPLE 

“The  record  of  North  American’s  P-5 1  Mustang  fighter  proves,  however,  that  it  is  both 
possible  and  practical  to  create  a  single  basic  design  that  can  be  modified,  as  military 
needs  dictate,  to  keep  abreast  of  requirements.”  (Nelson,  1944:127) 

“These  achievements  are,  from  an  engineering  standpoint,  remarkable — because  they 
were  accomplished  by  a  plane  that  does  not  to  any  extent  embody  previously  unknown 
engineering  features,  but  rather  employed  refinements  of  known  accepted  practices.” 
(Nelson,  1944:129) 

“It  is  an  extremely  simple  airplane  and  has  such  perfect  handling  qualities  as  to  put  a 
smile  of  joy  on  the  face  of  any  fighter  pilot.”  (Sanders,  1942) 

“The  problem  of  supply  of  a  new  airplane  is  not  as  great  as  it  first  seems  due  to  the  fact 
that  engines,  guns,  radios,  instruments,  and  man  [sic]  other  parts  are  the  same  as  those 
used  on  P-40s.”  (Sanders,  1942). 


TINY 

“The  P-5 1  is  requested  in  preference  to  the  P-47  because  of  its  smaller  size,  ease  of 
maintenance,  economy  of  operation,  range  and  because  many  of  the  accessories  for  it  are 
already  available  in  this  area  [Karachi  Air  Base,  India].”  (Sanders,  1942) 


OUTCOME 

“. . .  the  Mustang  has  been  credited  as  the  best  low-altitude  cooperational  craft,  the  most 
versatile  dive  bomber,  the  fastest  high-altitude  fighter,  and  the  plane  with  the  greatest 
range.”  (Nelson,  1944:127) 

“. . .  it  possessed  the  longest  range  of  any  single  engine  fighter  in  the  war.”  (Nelson, 
1944:129) 

“The  Mustang  has  literally  been  used  for  every  type  of  task  during  its  thirty- five  years  of 
service. . .  A  total  of  14,8 19  Mustangs  were  manufactured  by  North  American;  in  the 
European  Theater  of  Operations  the  Mustang  accounted  for  nearly  5,000  enemy  aircraft 
destroyed  in  aerial  combat  and  over  4,000  aircraft  destroyed  on  the  ground;  and  the 
Mustang  flew  over  213,800  combat  sorties  while  in  USAAF  service.  That  is  a  record  that 
is  unlikely  to  be  beaten  by  any  other  manned  fighter.”  (Quarterly  Review,  1975) 


ANALYSIS 

The  P-51  Mustang  is  one  of  the  most  famous  and  successful  fighter 
aircraft  of  the  WWII  era,  both  operationally  and  programmatically.  It  was  an  engineering, 
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logistical  and  operational  success,  and  sets  an  example  we  would  do  well  to  follow  even 
today.  The  P-51  was  designed  and  built  in  117  days,  which  was  the  first  of  several 
records  it  would  eventually  break.  Nearly  15,000  Mustangs  were  manufactured  over  the 
years,  and  countless  books  have  been  written  praising  the  Mustang’s  contributions  to 
national  defense. 

The  principles  that  guided  its  development  shine  light  on  several  important 
aspects  of  the  FIST  value  set,  specifically  the  use  of  mature  technologies.  Nelson  made  a 
point  of  bragging  that  the  Mustang  “does  not  to  any  extent  embody  previously  unknown 
engineering  features,  but  rather  employed  refinements  of  known  accepted  practices.” 
(Nelson,  1944:129).  This  emphasis  on  “known  accepted  practices”  and  studious 
avoidance  of  “unknown  engineering  features”  is  a  key  indicator  of  the  Simple  value. 

Pilots  made  a  similar  point  of  bragging  about  how  simple  the  aircraft  was  to  fly 
and  maintain.  By  relying  on  readily  available  systems  and  components,  North  American 
was  able  to  minimize  the  cost,  timeline,  complexity  and  maintenance  difficulties 
associated  with  the  new  aircraft.  As  Col  Sanders  explained  in  his  letter  from  Karachi  Air 
Base,  the  P-5 1  could  be  maintained  easily  with  existing  supplies  and  tools,  which  was  a 
significant  measure  of  merit,  particularly  in  such  an  austere  AOR.  The  Mustang’s 
outstanding  performance,  of  course,  was  equally  important,  but  a  high-performance 
aircraft  that  cannot  be  supported  and  supplied  in  a  remote  region  like  Karachi  would  have 
been  worse  than  useless. 

In  terms  of  the  Inexpensive  value,  the  Mustang  is  consistently  described  in  terms 
of  its  “economy  of  operation,”  and  praised  for  being  “economical  to  produce.”  Given  the 
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natural  trend  in  military  technology  toward  increasing  costs,  this  economy  is  most 
certainly  not  accidental  and  is  at  least  indirect  evidence  of  the  Inexpensive  value.  As  a 
measure  of  merit,  however,  the  Mustang’s  economy  is  an  implicit  value  rather  than  an 
explicit.  It  is  something  project  leaders  and  fliers  bragged  about,  but  not  as  strongly  as 
they  bragged  about  the  Mustang’s  operational  effectiveness.  Recall  that  what  project 
leaders  brag  about  is  usually  an  indicator  of  what  they  value,  and  so  the  Mustang  gets  a 
relatively  high  I-score. 
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PROJECT  NAME:  F-5  Freedom  Fighter  /  Tiger 
DATES:  1954  -  1959 

STORY  SOURCE:  Gentry,  1976;  AF  Museum  Fact  Sheet;  Burton,  1993;  FAS.org; 
Martin  &  Schmidt,  1987; 

RUBRIC  SCORE:  35 

F-Score:  5 
/-Score:  1 0 
S-Score:  10 
T- Score:  10 

OUTCOME:  A 
SAMPLE  STATEMENTS: 

FAST 

“The  F-5  first  flew  on  July  30th,  1959,  and  deliveries  to  the  Tactical  Air  Command  for 
instructing  foreign  pilots  began  in  April,  1964.”  (National  Museum  of  the  Air  Force  Fact 
Sheet) 


INEXPENSIVE 

Cost:  $756,000  (USAF  Museum  F-5  Fact  Sheet) 

“The  F-5  is  a  small,  low-cost  fighter  version  of  the  T-38  trainer. . .  Thousands  of  F-5’s 
have  been  sold  to  Third  World  countries  because  they  are  relatively  inexpensive.” 
(Burton,  1993:99) 

“The  F-5  can  be  operated  at  one  third  of  what  it  costs  to  operate  an  F/A-18.”  (FAS.org) 

“The  F-5  is  a  supersonic  fighter  combining  low  cost,  ease  of  maintenance  and  great 
versatility.”  (National  Museum  of  the  Air  Force  Fact  Sheet) 

“This  transition  to  the  F-5  adversary  aircraft  will  provide  Active  and  Reserve  Navy  pilots 
with  air-to-air  combat  training  at  significant  savings  to  the  taxpayer.”  (FAS.org) 


SIMPLE 

“The  Department  of  Defense  (DoD)  objective  was  to  develop  a  simple,  inexpensive  but 
reasonably  capable  fighter  which  could  be  provided  or  sold  to  those  countries  which 
could  not  afford,  or  lacked  the  capability  to  operate,  more  complex  aircraft.”  (Gentry, 
1976:5) 


138 


TINY 


Wingspan:  25  feet  10  inches  (USAF  Museum  F-5  Factsheet)  [The  F-16’s  wingspan  is  32 
ft  8  inches,  per  USAF  F-16  Factsheet] 

Weight:  8085  pounds  empty  (USAF  Museum  F-5  Factsheet)  [F-16  weighs  19,700 
without  fuel,  per  USAF  F-16  Factsheet] 

“In  1962,  the  Air  Force  had  awarded  Northrop  a  contract  to  produce  the  F-5  Freedom 
fighter,  a  very  lightweight  (13,000-pound),  twin-engined  tactical  fighter...”  (Gentry, 
1976:5) 

OUTCOME 

“Approximately  1 100  were  delivered  to  15  countries  over  an  eight-year  period.”  (Gentry, 
1976:5) 

“The  F-5  is  a  supersonic  fighter  combining  low  cost,  ease  of  maintenance  and  great 
versatility.  More  than  2,000  F-5  aircraft  have  been  procured  by  the  USAF  for  use  by 
allied  nations.”  (USAF  Museum  F-5  Fact  Sheet) 

“Northrop’s  F-5  program  produced  a  highly  lucrative  international  fighter  and  a  relatively 
successful  aircraft  by  most  measures.  Thirty-two  countries  have  purchased  more  than 
2600  F-5’s  with  28  different  configurations  of  the  F-5E  alone.”  (Martin  &  Schmidt, 
1987:12) 

“The  F-5...  is  suitable  for  various  types  of  ground-support  and  aerial  intercept  missions, 
including  those  which  would  have  to  be  conducted  from  sod  fields  in  combat  areas.” 
(USAF  Museum  F-5  Factsheet). 


ANALYSIS 

The  F-5  is  one  of  the  few  modern  aircraft  designed  explicitly  around  the  FIST 
values.  Northrop  focused  on  developing  a  small  fighter  that  would  be  inexpensive  and 
simple,  in  terms  of  both  ownership  and  operations,  so  the  Inexpensive  and  Simple  values 
are  the  most  explicit.  This  combination  of  simplicity  and  low  cost  was  the  plane’s  main 
attraction  for  prospective  buyers.  It  was  easily  transformed  into  several  variants,  further 
evidence  of  the  flexibility  inherent  in  simple  designs. 

The  Fast  value  was  less  explicitly  held,  at  least  as  expressed  in  the  stories, 
interviews  and  case  studies  reviewed  in  this  project.  However,  Fast  was  clearly  an 
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implicit  value,  as  evidenced  by  the  program’s  development  timeline  and  sense  of  urgency. 
Further,  the  emphasis  on  keeping  costs  low  seems  to  necessitate  a  short  development 
timeline,  since  excessive  delays  would  increase  the  cost  of  the  system. 

The  F-5’s  operational  success  is  impressive.  It  is  widely  used  in  other  countries  as 
a  primary  jet  fighter,  and  as  an  aggressor  aircraft  by  the  US  military  (Air  Force  and 
Navy).  The  US  uses  a  trainer  version  of  the  F-5,  the  T-38  Talon,  to  train  Air  Force  and 
NATO  pilots,  as  well  as  NASA  astronauts.  NASA  also  uses  it  as  a  chase  plane  to  support 
shuttle  missions. 
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PROJECT  NAME:  A-10  Thunderbolt  II 

DATES:  1967  -  1975 

STORY  SOURCE:  Burton,  1993;  USAF  A-10  Fact  Sheet;  Doscher,  2008;  Jacques  & 
Strouble,  2008;  Dahl,  2003. 

RUBRIC  SCORE:  35 

F -Score:  10 
I-Score:  10 
S-Score:  10 
T- Score:  5 

OUTCOME:  A 
SAMPLE  STATEMENTS: 

FAST 

“The  first  production  A-10A  was  delivered  to  Davis-Monthan  Air  Force  Base,  Ariz.,  in 
October  1975.”  (USAF  A-10  Fact  Sheet) 

“The  RAD  [Requirements  Action  Directive]  for  the  A-X  . . .  cited  an  urgent  need  (IOC  of 
1970)  which  dictated  maximum  employment  of  existing  state-of-the-art  technology  in  its 
design,  thus  allowing  for  a  compressed  conceptual  design  phase.”  (Jacques  &  Strouble, 
2008:17-18) 

“The  required  IOC  of  December  1970  was  considered  high  risk  with  respect  to  cost  and 
schedule,  but  achievable  if  the  concept  definition  phase  was  reduced  to  a  four  month 
contract  definition  followed  by  competitive  source  selection...”  (Jacques  &  Strouble, 
2008:21) 

“The  change  to  a  competitive  prototyping  approach  had  a  significant  effect  on  IOC,  with 
the  new  projected  IOC  now  in  FY76.”  (Jacques  &  Strouble,  2008:30) 

“Contractors  would  have  3  months  to  respond  to  the  RFP  and  the  government  intended  to 
complete  its  evaluation  of  proposals  and  make  award  within  a  75  day  period.”  (Jacques  & 
Strouble,  2008:32) 

“The  Air  Force  also  believed  the  A-10  to  be  closer  to  production,  thus  allowing  for  faster 
progress...”  (Jacques  &  Strouble,  2008:39) 


INEXPENSIVE 

“These  shells  [fired  by  an  A-10]  are  very  effective  at  destroying  tanks,  trucks,  bunkers, 
and  most  targets  of  interest  on  a  battlefield.  They  cost  only  $13  apiece,  a  far  cry  from 
$100,000  for  a  maverick  missile.”  (Burton,  1993:25) 

“The  cheapest  airplane  to  buy  and  operate,  it  was  also  the  most  effective  ground  support 
airplane  in  the  world.”  (Burton,  1993:26) 
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“The  specifications  also  demanded  that  the  aircraft  cost  less  than  $3  million.”  (Coram, 
2002:236) 

“Usually  there  are  no  cost  constraints  on  an  aircraft-design  program. . .  In  all  the  history 
of  the  Air  Force,  the  A-X  was  the  single  exception.  It  had  to  be  cheap.  It  had  to  cost  less 
than  the  Cheyenne.”  (Coram,  2002:234) 

“In  a  force  cost  analysis,  the  A-X  had  the  lowest  total  force  cost  due  to  the  small  force 
required,  which  could  be  traced  to  high  availability  over  the  battlefield,  high  sortie  rates 
and  high  sortie  effectiveness.”  (Jacques  &  Strouble,  2008:27) 

“In  order  to  reduce  these  costs,  the  Air  Force  was  to  provide  guidance  stressing 
simplicity  of  design,  ease  of  maintenance,  and  the  importance  of  keeping  costs  of  the  A- 
X  to  a  minimum. . .  In  order  to  achieve  cost  savings,  the  specified  performance 
requirements  were  not  to  be  considered  firm  specifications  but  goals...”  (Jacques  & 
Strouble,  2008:33) 


SIMPLE 

“The  nose  of  the  plane  [A- 10]  is  pointed  at  the  target  and  a  burst  of  cannon  shells 
squeezed  off — simply,  quickly  and  efficiently.”  (Burton,  1993:25) 

“The  A-X  was  to  use  an  existing  state-of-the-art  engine  in  order  to  achieve  an  early  IOC 
[Initial  Operating  Capability] . . .  The  A-X  would  also  use  existing  state-of-the-art 
equipment  for  avionics.”  (Jacques  &  Strouble,  2008:18) 

“The  avionics  for  the  A-X  were  specified  in  tenns  of  a  ‘skeleton’  package  (below 
minimum  requirements),  a  ‘lean’  package  (met  only  minimum  requirements),  and  three 
add-on  packages  that  would  supplement  the  ‘lean’  package.”  (Jacques  &  Strouble, 
2008:20) 

“. . .  it  was  intended  that  simplicity  of  design  would  lead  to  a  shorter  development  time, 
lower  life  cycle  cost,  reduced  maintenance  times,  increased  sortie  rates  and  the  ability  to 
operate  from  austere  bases.”  (Jacques  &  Strouble,  2008:22) 

“In  October  1969,  Secretary  of  the  Air  Force  Harold  Seamans  chose  an  alternative  that 
was  tenned  ‘Parallel  Undocumented  Development.’  This  approach  would  require  a 
minimal  amount  of  documentation  during  the  competitive  prototyping  phase  to  encourage 
innovation  and  initiative  on  the  part  of  the  contractors.”  (Jacques  &  Strouble,  2008:30) 


TINY 

“Dr.  John  Foster  [DDR&E]  stated  that  ‘the  proposed  aircraft  seems  to  be  too  large. . .  A 
smaller,  less-costly,  quick  reaction  aircraft  seems  more  appropriate.’”  (Jacques  & 
Strouble,  2008:29) 

“The  representative  A-X  from  DCP-23A  [Development  Concept  Paper]  was  slightly 
smaller,  lighter  and  cheaper  than  that  of  DCP-23.”  (Jacques  &  Strouble,  2008:30) 
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“In  keeping  with  the  philosophy  of  minimum  documentation  for  the  Parallel 
Undocumented  Development  phase,  the  SPO  [System  Program  Office]  was  to  be  manned 
on  an  austere  basis.”  (Jacques  &  Strouble,  2008:32) 

“The  A-X  request  for  proposal  (RFP),  including  all  attachments  and  ‘boiler  plate’  was 
104  pages,  and  it  limited  each  contractor’s  response  to  585  pages.  This  represented  a 
sizeable  reduction  in  the  RFP  for  its  time. . .”  (Jacques  &  Strouble,  2008:32) 


OUTCOME 

“The  A- 10s  turned  out  to  be  one  of  the  true  success  stories  of  the  [first  Gulf]  war.  They 
represented  only  15  percent  of  the  combat  aircraft  in  the  war,  yet,  according  to  the  Air 
Force  Times,  ‘flew  about  a  third  of  the  total  sorties  and  were  responsible  for  more  than 
half  the  confirmed  bomb  damage’  against  the  enemy.  Put  another  way,  the  A- 10  was 
responsible  for  more  damage  to  the  enemy  than  all  the  other  combat  aircraft  put 
together...”  (Burton,  1993:242) 

“During  approximately  8,624  combat  sorties,  only  six  A- 10s  were  lost.  Two  of  them 
crashed  while  landing,  which  resulted  in  one  of  the  two  pilot  fatalities  that  occurred.” 
(Burton  1993:  242) 

“The  A- 10C  is  already  saving  lives.  The  upgraded  Warthog  has  flown  nearly  1,000 
combat  sorties  since  it  came  online  in  August,  providing  combat  air  support  to  U.S.  and 
NATO  forces  in  Afghanistan.”  (Doscher,  2008) 


ANALYSIS 

The  development  of  the  A- 10  Thunderbolt  II  (aka  the  Warthog)  was  strongly 
guided  by  the  FIST  value  set,  both  out  of  necessity  and  by  the  designer’s  philosophical 
preferences.  It  is  no  secret  that  the  Air  Force  historically  has  not  placed  a  high  priority  on 
the  Close  Air  Support  (CAS)  mission  area,  and  during  “the  1950’s  and  early  1960’s,  the 
USAF  made  no  effort  to  field  an  aircraft  specifically  designed  for  CAS”  (Dahl,  2003:2), 
preferring  instead  the  more  glamorous  missions  of  air  superiority  and  long-range  strike. 
Thus,  project  leaders  strove  to  keep  a  low  profile  for  the  A- 10  (particularly  at  the 
project’s  start),  by  delivering  quickly  and  at  a  low  cost. 

Few  systems  demonstrated  the  unified  nature  of  the  FIST  values  as  strongly  as  the 
A- 10.  Descriptions  of  the  project  leaders’  priorities  and  values  consistently  use  phrases 
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like  “simplicity  of  design  would  lead  to  a  shorter  development  time,  lower  life  cycle 
cost...”  and  “in  order  to  reduce  costs,  the  Air  Force  stressed  simplicity  of  design...”  This 
sometimes  makes  it  difficult  to  isolate  which  of  the  four  FIST  components  was  the 
driving  factor,  perhaps  because  no  single  aspect  of  FIST  had  priority.  The  A-X  project 
leaders  valued  the  entire  FIST  set,  as  an  indivisible  priority,  not  as  a  set  of  attributes  to  be 
selected  piecemeal  in  trade-off  analyses. 

The  A- 10  specifications  required  a  low  cost  system,  which  drove  the  designers  to 
use  existing,  mature  systems  and  technologies,  which  lead  to  a  shortened  development 
schedule.  The  project  leaders  were  staunch  advocates  of  simplicity,  both  in  the  system’s 
design  and  operation  as  well  as  organizationally.  They  explicitly  preferred  the  low-cost, 
simple  munitions  fired  by  the  A-lO’s  gun  over  the  expensive  high-tech  alternatives. 

The  A- 10  is  one  of  the  most  operationally  successful  aircraft  in  the  USAF 
inventory,  and  stories  of  its  resilience  are  legendary.  As  Burton  pointed  out,  although  the 
A- 10  only  represented  15%  of  the  combat  aircraft  in  theater  during  the  first  Gulf  War,  it 
was  “responsible  for  more  damage  to  the  enemy  than  all  the  other  combat  aircraft  put 
together. . .”  (Burton,  1993:242).  It  doesn’t  get  much  better  than  that. 
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PROJECT  NAME:  F-20  Tigershark 
DATES:  -  1986 

STORY  SOURCE:  Jurkus,  1990;  Martin  &  Schmidt,  1987; 

RUBRIC  SCORE:  35 

F-Score:  5 
I-Score:  10 
S-Score:  10 
T-  Score:  10 

OUTCOME:  Failure  -  zero  sales. 

SAMPLE  STATEMENTS: 

FAST 

“The  rollout  of  the  first  F-20  Tigershark  occurred  32  months  after  the  program  go- 
ahead — it  came  out  a  month  ahead  of  schedule.”  (Martin  &  Schmidt,  1987:5) 

“Northrop  rapidly  tested  the  Tigershark  through  its  flight  envelope  in  order  to  enable 
foreign  air  force  pilots  to  fly  the  aircraft  as  soon  as  possible  for  marketing  reasons.” 
(Martin  &  Schmidt,  1987:6) 


INEXPENSIVE 

“The  strategy  of  Northrop  with  respect  to  the  F-20  was  to  design,  produce  and  develop  an 
intennediate  export  fighter  in  accordance  with  United  States  FX  [Foreign  Export]  policy 
for  an  export  market  estimated  to  be  in  the  thousands. . .  It  was  designed  to  be  less 
expensive  to  buy,  own  and  operate.”  (Jurkus,  1990:61) 

“The  cost  of  ownership  and  the  expense  of  owning  and  operating  the  F-20  was  less  than 
half  the  cost  of  an  F-16,  according  to  Northrop.”  (Jurkus,  1990:62) 

“It  is  evident  that  from  early  on  in  the  program,  Northrop  was  concerned  over  the  cost- 
effective  producibility  of  the  Tigershark.  They  took  great  steps  towards  ensuring  that  they 
would  be  able  to  produce  a  quality  product  that  could  be  sold  for  a  reasonable  price.” 
(Martin  &  Schmidt,  1987:9) 

“The  proposed  fixed  price  for  the  F-20s  was  $15  million  with  support  (1985  dollars), 
substantially  less  than  GD’s  going  price  of  $18  million  for  the  F-16C.”  (Martin  & 
Schmidt,  1987:17) 


SIMPLE 

“Northrop’s  export  fighter  program  has  been  characterized  by  a  consistent  philosophy.  In 
designing  and  developing  an  aircraft,  performance  must  not  outweigh  cost,  reliability, 
maintainability,  and  operability. . .  Their  typical  client  country  hasn’t  had  the  kind  of 
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support  structure  to  operate  and  maintain  a  ‘Ferrari’  level  aircraft,  but  has  done  well  with 
a  ‘Ford  Escort’  aircraft”  (Martin  &  Schmidt,  1987:3) 

“A  corollary  to  the  Northrop  philosophy  stated  above  is  a  willingness  to  give  up  a  little 
extra  performance  in  order  to  make  significant  gains  in  tenns  of  affordability,  reliability, 
maintainability,  and  operability.  Northrop  had  never  pushed  the  edge  of  the  technical 
‘envelope’  with  the  F-5  program.  They  did  not  intend  for  the  Tigershark  to  be  an 
exception.  Northrop  put  these  principles  into  practice  by  selecting  F-20  subsystems  which 
were  either  proven,  or  which  had  good  paper  ‘specs’  with  respect  to  reliability.”  (Martin 
&  Schmidt,  1987:4) 

“The  F-20  also  met  its  ‘operability’  goal:  foreign  pilots  were  able  to  fly  the  single  seat  F- 
20  after  just  two  days  of  simulator  training.”  (Martin  &  Schmidt,  1987:6) 


TINY 


OUTCOME 

“After  five  years,  one  billion  dollars  in  development  costs,  and  strong  lobbying  and 
marketing  efforts,  Northrop  Corporation’s  F-20  Tigershark  was  recently  laid  to  rest  with 
little  fanfare.”  (Jurkus,  1990:59) 

“To  the  trained  eyes  of  prospective  buyers  its  perfonnance  was  dismal.”  (Jurkus,  1990:62) 

“Northrop  and  many  observers  claimed  that,  from  a  technical  point  of  view,  the  program 
was  a  resounding  success.  In  point  of  fact,  no  one  can  know.  No  F-20’s  were  ever  sold.” 
(Martin  &  Schmidt,  1987:1) 

“Northrop  considers  the  F-20  program  to  have  been  a  great  R&D  success.  They  are 
correct  in  the  sense  that  theF-20  met  or  exceeded  Northrop  criteria  for  performance, 
reliability,  maintainability  and  operability.”  (Martin  &  Schmidt,  1987:3) 

“The  F-20  accomplished  approximately  1500  flights  before  the  program  was 
terminated. . .  On  the  basis  of  its  1500  flights,  the  F-20  Tigershark  seemed  to  score  rather 
well.  It  had  a  much  faster  scramble  time  than  the  F-16. . .  The  F-20  was  inferior  to  the  F- 
16  in  operating  range  (10-15  percent  less  range  depending  on  altitude,  speed,  etc.)  and  in 
the  amount  of  armaments  that  could  be  carried.”  (Martin  &  Schmidt,  1987:6) 

“The  MFHBF  [Mean  Flight  Hours  Between  Failures]  for  the  F-20  was  4.2  at  the  time  of 
program  tennination.  The  F-16  had  accumulated  a  3.2  MFHBF  over  many  years  of 
operational  use.  The  F-20  required  maintenance  manhours  per  flight  hour  (MMH/FH0  of 
15.1  during  its  test  run.  The  F-16  had  accumulated  a  MMH/FH  of  33.9.”  (Martin  & 
Schmidt,  1987:6) 
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ANALYSIS 

Following  their  success  with  the  F-5,  Northrop  explicitly  focused  on  building  a 
FISTy  aircraft.  This  is  seen  by  their  reliability  on  proven  technology  and  multiple 
decisions  to  sacrifice  perfonnance  in  order  to  make  gains  in  affordability,  reliability, 
maintainability  and  operability.  The  literature  did  not  make  many  explicit  statements  in 
support  of  the  Fast  value,  but  given  the  importance  of  simplicity  and  cost,  along  with  the 
potentially  large  foreign  market,  Fast  seems  to  be  an  implicit  value. 

The  Reagan  administration’s  change  to  the  previous  Carter  administration’s 
position  on  foreign  weapon  sales  forced  the  F-20  to  compete  against  the  fully-configured 
F-16,  instead  of  against  the  degraded  version  approved  for  foreign  sale  under  Carter.  As 
soon  as  foreign  buyers  had  the  option  to  purchase  the  F-16,  the  F-20’s  appeal  was  much 
diminished. 

It  is  interesting  to  note  that  the  F-16  was  originally  the  epitome  of  the  FIST  model, 
but  when  later  versions  of  the  F-16  are  put  alongside  the  new  F-20,  the  Falcon  ends  up 
looking  expensive,  complex,  etc.  This  is  not  just  a  question  of  comparative  attributes 
between  the  F-16  and  the  F-20,  but  rather  it  indicates  a  change  in  the  F-16’s  design 
philosophy.  As  Burton’s  analysis  of  later  versions  of  the  Falcon  explains,  “This  beautiful 
little  aircraft  [the  F-16],  which  had  been  the  best  dogfighter  in  the  world,  was  now  just 
another  overpriced,  overweight,  underperforming  monument  to  high  technology.” 
(Burton,  1993:103-104).  So,  a  system’s  FIST  score  is  not  only  relative,  it  is  also  not  static. 

Martin  &  Schmidt  point  out,  “Now  that  F-16A’s  had  been  sold  to  Venezuela  and 
Pakistan,  most  developing  countries  began  requesting  similar  aircraft,  regardless  of  their 
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budgets  or  realistic  assessments  of  their  security  threats.”  (Martin  &  Schmidt,  1987:14- 
15).  Rather  than  base  their  purchase  decision  on  an  assessment  of  the  actual  threat 
environments  they  faced,  foreign  governments  preferred  the  larger,  more  expensive,  more 
complex  F-16,  and  they  “demanded  the  highest  level  of  technology  they  could  acquire.” 
(Martin  &  Schmidt,  1987:21). 

This  was  a  psychological  and  emotional  decision  on  the  part  of  foreign 
governments,  based  on  a  desire  to  appear  fully  modern  and  tough.  It  should  be  pointed 
out  that  much  the  same  decision  process  goes  on  in  the  US  as  well,  so  they  were  only 
following  the  DoD’s  example.  In  the  end,  the  very  values  which  led  to  the  development 
of  the  Tigershark  as  a  simple,  reliable,  low-cost  fighter  jet  actually  undercut  its 
marketability,  because  these  values  were  not  shared  by  the  potential  customers. 

The  F-20  program  was  a  resounding  failure — Northrop  was  not  able  to  sell  even  a 
single  F-20.  The  Tigershark’s  limited  perfonnance  was  certainly  a  contributing  factor,  as 
was  the  change  in  US  policy  on  weapon  exports.  However,  the  primary  reason  appears  to 
be  the  striking  mismatch  between  Northrop ’s  design  values  and  the  values  of  its  potential 
clients. 


CO-EVALUATOR  ANALYSIS 
RUBRIC  SCORE:  30 

F-Score:  0 
I-Score:  10 
S-Score:  10 
T-  Score:  10 

FAST :  I  didn’t  detect  any  value  or  "stated  preference  and  priorities"  with  regard  to  being 
fast.  Although  the  case  study  reported  the  program  accomplished  its  first  rollout  a  month 
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ahead  of  schedule,  that  is  an  outcome  and  not  necessarily  a  product  of  having  a  "fast" 
value.  For  example,  being  fast  could  actually  be  a  product  of  trying  to  be  simple. 
Therefore  I  gave  it  a  score  of  0. 

Inexpensive:  You  couldn’t  read  this  case  study  without  coming  away  knowing  that 
developing  an  inexpensive  product  was  a  HUGE  driving  factor.  From  the  beginning 
when  CEO  Tom  Jones  released  his  basic  philosophy,  affordability  was  included  while  the 
other  measures  (reliability  and  maintainability)  were  products  of  wanting  to  have 
affordable  life-cycle  costs.  Additionally  they  made  specific  design  decisions  to  remain 
affordable.  Therefore  I  gave  it  a  score  of  10. 

Simple:  Northrop  made  choices  for  the  purpose  of  decreasing  complexity.  A  great 
example  of  this  was  the  choice  of  engine  that  would  have  half  the  parts  of  the  F-16 
engine.  Also,  they  aimed  to  use  proven  technologies.  Although  I  think  they  valued 
simplicity,  it  wasn’t  simplicity  for  simplicity  sake.  It  was  simplicity  for  reliability  and 
maintainability  sake.  However,  to  detennine  if  they  value  something  highly  I  don't 
believe  the  purpose  of  why  they  value  it  is  necessarily  important,  what  is  important  is 
they  did  value  it  and  made  decisions  based  on  it;  I  gave  this  a  score  of  10. 

Tiny:  I  was  on  the  fence  with  this  one.  I  didn't  see  any  explicit  comments  about  being 
tiny  or  small.  For  this  one  I  had  to  look  at  behavior  and  infer  what  they  valued  based  on 
it.  While  there  weren't  "Strong,  explicit  affirmation  of  the  importance  of  small"  I  believe 
there  was  "concrete  steps  taken  to  actually  reduce  size".  These  include  the  redesigning  of 
the  production  center  to  decrease  the  size  and  bureaucracy  of  production,  also  there  was 
comments  about  the  small  size  of  documentation.  The  case  study  made  it  clear  there  was 
less  documentation  than  what  would  have  accompanied  a  government  sponsored  program 
and  the  initial  CEO  design  philosophy  was  kept  to  just  one  page.  I  didn’t  believe  these 
were  "modest  and  infonnal  commitments"  so  I  therefore  gave  it  a  score  of  10. 

Outcome:  Without  a  doubt  the  outcome  of  this  program  falls  squarely  in  the  "Mission 
Failure"  category.  The  system  was  rejected  by  users  and  the  program  cancelled  before 
delivery.  Score  F 

Additional  Comments :  At  first  after  reading  the  case  study  I  thought  I  would  have  to 
score  this  program  twice.  Once  for  pre -Reagan  arms  sales  policy  and  once  for  post- 
Reagan  arms  sales  policy.  This  is  because  program  behavior  changed  dramatically  after 
it  was  decided  they  were  competing  against  the  F-16.  However,  I  ultimately  decided  that 
even  though  their  actions  changed  their  values  didn’t.  I  got  the  impression  they  still 
wanted  to  stay  as  inexpensive,  simple,  and  tiny  as  possible  while  adding  some  more 
capability  to  compete  better.  Therefore,  I  determined  it  was  fair  to  grade  this  only  once. 

Your  warning  on  the  "instruction  to  Co-investigators"  page  to  not  focus  on  programmatic 
success  but  to  focus  on  operational  outcome  was  very  important  for  this  particular  case 
study.  By  all  indications  this  was  a  program  that  produced  a  good  airplane  on  schedule 
and  budget  while  being  affordable.  But  that  just  wasn’t  enough.  While  Northrop  valued 
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FIST,  the  world  didn’t.  The  world  didn’t  want  to  evaluate  their  threat  environment  and 
detennine  the  best  airplane  to  meet  that  threat  while  weighing  performance  vs.  cost. 

They  wanted  the  best,  period.  They  wanted  what  the  U.S.  was  flying  whether  that  was  in 
their  interest  or  not.  I  think  of  those  program  managers,  engineers  and  others  at  Northrop 
who  did  so  much  right  but  still  failed  and  it  reminds  me  how  uncertain  anything  is  in  this 
business. 
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PROJECT  NAME:  A-l  /  AD  Skyraider 

Note  on  names:  the  BT2D-1  became  the  AD-1,  which  was  eventually  christened  the 
Skyraider.  Crews  referred  to  the  AD  as  the  “Able  Dog.  ” 

DATES:  1944  -  1957 

STORY  SOURCE:  USAF  Museum  Fact  Sheet:  Douglas  A-1E  Skyraider;  Johnson,  2008; 
Boyne,  2007 

RUBRIC  SCORE:  30 

F -Score. TO 
1-Score:  5 
S-Score:  1 0 
T -Score: 5 

OUTCOME:  A 
SAMPLE  STATEMENTS: 

FAST 

“The  Navy  men  liked  everything  about  the  idea  except  the  extra  30  days;  They  informed 
Heinemann  they  wanted  to  see  a  presentation  of  his  new  design  proposal  at  9  the  next 
morning!  In  truth,  Heinemann  wasn’t  wholly  unprepared.  He  and  his  top  staff  members, 
Leo  Devlin  and  Gene  Root,  had  for  several  weeks  been  sketching  out  ideas  for  a  totally 
new  design.”  (Johnson,  2008:2) 

“The  presentation  was  over  by  late  morning,  and  the  three  men  were  told  to  keep  their 
seats  and  wait.  By  noon  they  got  an  answer:  Douglas  was  authorized  to  cancel  the  BTD 
program  and  fund  construction  of  25  preproduction  examples  of  the  proposed  model,  the 
XBT2D-1.  BuAer  [Navy  Bureau  of  Aeronautics]  gave  them  exactly  nine  months  to  get 
the  plane  in  the  air.  When  Heinemann  returned  to  the  El  Segundo  plant,  his  instructions 
to  his  staff  and  employees  were  terse:  ‘Nothing  must  interfere  with  the  completion  of  this 
aircraft  on  schedule.’”  (Johnson,  2008:2) 

“On  March  19,  1945  -  almost  nine  months  to  the  day  from  Heinemann’ s  meeting  in 
Washington  -  the  first  XBT2D-1  lifted  off  the  runway  at  El  Segundo.  Such  was  the  rush 
that  the  plane  had  flown  with  landing  gear  struts  and  wheels  borrowed  from  a  Vought 
Corsair  and  an  older  version  of  the  R-3350  engine. . .”  (Johnson,  2008:2) 

“Heinemann  and  his  El  Segundo  staff  moved  at  a  breakneck  pace  to  address  each  fault 
identified  by  NATC.”  (Johnson,  2008:3) 

“. . .  a  little  over  19  months  after  its  first  flight,  BuAer  declared  the  AD-1  ready  to  join  the 
fleet.  Heinemann  had  not  only  caught  up  with  Martin,  he  was  miles  ahead.”  (Johnson, 
2008:3) 

INEXPENSIVE 
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SIMPLE 

“Though  using  the  same  Wright  R-3350  power  plant,  it  was  a  far  simpler  design. . 
(Johnson,  2008:2) 

. .  the  XBT2D-1  was  less  complex  overall,  and  thus  cheaper  to  build  and  easier  to 
maintain  [than  competing  aircraft  from  Martin,  Curtiss  and  Kaiser].” 

“According  to  his  logbook,  he  [Capt  William  Smith,  USMC)  received  4.3  hours’ 
checkout  and  familiarization  time,  after  which  he  immediately  began  flying  combat 
sorties  in  interdiction  of  enemy  supply  lines  and  close  air  support  of  UN  troops.” 
(Johnson,  2008:1) 

“The  Skyraider  made  its  first  flight  in  March  1945,  and  for  the  next  12  years  Douglas 
continually  improved  it  through  several  versions.”  (USAF  Museum  Fact  Sheet) 

“...  could  it  be  routinely  and  safely  operated  from  carriers  by  ‘nugget’  aviators  (i.e. 
inexperienced  ensign  and  junior  grade  pilots  on  their  first  cruise)?  The  new  AD’s  passed 
this  test  with  flying  colors,  as  every  squadron,  each  with  its  share  of  nuggets,  completed 
carrier  qualifications  without  serious  incident.”  (Johnson,  2008:3) 


TINY 


OUTCOME 

“The  airplane  quickly  became  popular  with  pilots  and  maintenance  crews,  and  they  soon 
took  to  calling  it  the  ‘Able  Dog.’  ...  By  comparison,  the  introduction  of  Martin  AM-1  ’s 
to  squadron  service  in  1948  was  marked  by  frequent  accidents  and  excessive  maintenance, 
and  by  late  1949  the  remaining  examples  of  the  151  Maulers  built  were  replaced  by  new 
Ads.” 

“Because  of  its  ability  to  carry  large  bomb  loads,  absorb  heavy  ground  fire  and  fly  for 
long  periods  at  low  altitude,  the  A- IE  was  particularly  suited  for  close-support  missions.” 
(USAF  Museum  Fact  Sheet) 

“. . .  after  that  first  week  there  was  no  question  in  my  mind  that  our  AD’s  were  the  best 
planes  in  the  world  for  the  job  expected  of  us. . .  Even  after  all  these  years  of  progress,  I 
believe  the  AD  is  still  the  best  airplane  ever  made  for  close-in  attack  option. . .  better,  in 
fact,  than  anything  flying  today.”  [former  U.S.  Marine  Corps  Captain  William  C  Smith, 
who  flew  with  Marine  Attack  Squadron  121  in  the  Korean  War]  (Johnson,  2008:1) 

“Overall  perfonnance  and  handling  characteristics  were  rated  as  ‘exceeding 
expectations.’”  (Johnson,  2008:3) 

“When  NATC  [Naval  Air  Test  Center]  resumed  evaluations  of  newly  modified  AD-l’s  in 
the  fall  of  1946,  test  pilot  reports  were  highly  enthusiastic.  . .  .the  plane  was  graded  as  the 
best  dive-bombing  platform  NATC  had  ever  tested.  Equally  important,  NATC  regarded 
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the  AD-1  as  above  average  in  terms  of  maintainability  and  logistical  support  required.” 
(Johnson,  2008:3) 

“Ads  were  the  only  planes  capable  of  delivering  2,000-pound  bombs  with  dive-bomber 
precision  against  hard  targets  like  mountain  bridges  and  hydroelectric  dams.”  (Johnson, 
2008:4) 

“A-l’s  were  judged  the  best  planes  in  Southeast  Asia  for  escorting  troop-laden 
helicopters.”  (Johnson,  2008:5) 

“Though  never  intended  for  air-to-air  confrontations,  two  A-lH’s. . .  share  in  the  shoot- 
down  of  a  North  Vietnamese  MIG- 17  on  June  20,  1965.  Then  on  October  9,  1966,  Lt.  j.g. 
W.  Thomas  Patton  of  VA-176,  flying  an  A-1H  from  USS  Intrepid,  sent  another  MiG-17 
down  in  flames  near  Hanoi.”  (Johnson,  2008:5) 


ANALYSIS 

The  Douglas  A-l  Skyraider  development  story  offers  a  significant  amount  of 
evidence  that  the  project  leaders  held  to  the  FIST  values.  To  begin  with,  the  design  and 
development  was  accomplished  at  a  breakneck  pace.  In  fact,  according  to  some  reports, 
the  Skyraider  was  designed  in  a  single  night  at  the  Sattler  Hotel  in  Washington  DC. 
(Boyne,  2007:88).  The  truth  is  the  design  work  actually  began  “several  weeks”  before 
that  fateful  night,  but  whether  measured  in  weeks  or  hours,  the  design  was  completed  in 
record  time.  In  fact,  this  part  of  the  story  highlights  one  of  the  “tricks”  to  finishing  early  - 
start  early. 

While  the  Navy  only  gave  the  Douglass  team  a  single  night  to  produce  the  design, 
the  designers  were  able  to  draw  from  existing  designs  already  under  development,  which 
they  had  begun  in  anticipation  of  the  need.  The  end  result  was  an  ability  to  deliver  a 
combat  aircraft  design  proposal  in  less  than  24  hours...  and  to  deliver  the  first  prototypes 
in  less  than  nine  months. 

In  order  to  produce  the  design  and  the  prototypes  in  such  short  order,  the  small 
project  team  necessarily  pursued  a  design  philosophy  that  valued  simplicity.  This 


153 


preference  for  speed  and  simplicity  is  clearly  in  evidence  when  we  read  that  Douglass 
spent  the  next  twelve  years  updating  and  improving  the  aircraft.  That  is,  they  deemed  it 
more  important  to  deliver  a  first-generation  aircraft  on  time  than  to  deliver  a  fully- 
capable,  “perfect”  aircraft.  They  did  not  allow  their  design  modifications  and  tweaking  to 
interfere  with  the  need  to  deliver  actual  hardware,  and  thus  they  maintained  a  strict  focus 
on  being  Fast  and  Simple. 

The  team’s  emphasis  on  simplicity  was  also  in  evidence  by  the  fact  that  pilots 
with  minimal  training  could  fly  the  Skyraider  with  confidence.  Even  more  telling  is  the 
fact  that  the  aircraft  was  popular  with  maintenance  crews. 

The  remaining  two  FIST  values  -  Inexpensive  and  Tiny  -  are  less  in  evidence. 
This  is  partly  due  to  the  relative  dearth  of  data  on  the  Skyraider,  which  was  developed 
over  60  years  ago.  But  while  the  few  sources  did  not  spend  much  time  explicitly 
discussing  Inexpensive  and  Tiny,  there  are  some  clues  that  the  project  leaders  did  indeed 
hold  to  these  values.  However,  these  values  were  apparently  implicit,  not  explicit,  and 
thus  the  Skyraider’ s  1-score  and  T-score  are  lower. 

The  Skyraider’s  operational  record  is  as  impressive  as  they  come.  Beloved  by 
pilots  and  maintainers  alike,  it  has  a  long  record  of  distinguished  service  in  WW11,  Korea 
and  Vietnam  -  even  including  the  shoot-down  of  a  couple  MiG- 17  jet  fighters  (an 
impressive  accomplishment  for  a  propeller-driven  bomber),  ft  is  clearly  a  successful 
aircraft. 
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PROJECT  NAME:  XP-75  /  P-75  Eagle 
DATES:  1942-1945 

STORY  SOURCE:  Holley,  1987;  Thornton,  1946; 

RUBRIC  SCORE  25 

F -Score:  10 
I-Score:  10 
S-Score:  5 
T -Score:  0 

OUTCOME:  F  -  project  cancelled 
SAMPLE  STATEMENTS: 

FAST 

.  the  object  of  this  development  was  to  provide,  with  a  minimum  of  delay,  a  fighter 
type  airplane  having  exceptional  climb  perfonnance.  Engineering  and  production  was  to 
be  expedited...”  (Thornton,  1946:1) 

“Engineering  work  proceeded  as  rapidly  as  possible  throughout  the  winter  of  1942-1943 
and  spring  and  early  summer  of  1943.”  (Thornton,  1946:2) 

“In  September  1941,  the  corporation  [General  Motors]  approached  Maj  Gen  O.P.  Echols, 
the  commanding  general  of  Materiel  Command,  with  a  proposal  to  develop  a  fighter 
aircraft  in  a  remarkably  short  time  by  using,  as  far  as  possible,  structures,  controls,  and 
accessories  already  in  full  production  for  other  aircraft  in  order  to  obviate  the  need  for 
long  delays  in  tooling  up  for  production.”  (Holley,  1987:587) 

“Only  the  main  fuselage  itself  would  have  to  be  developed  anew.  By  such  short-cuts,  the 
plane  was  expected  to  be  ready  for  flight  testing  in  six  months.”  (Holley,  1987:587) 

“. . .  the  air  force  [v/t  ]  took  steps  to  shorten  the  delays — first,  by  increasing  the  number  of 
experimental  models  to  be  procured  from  two  to  eight  to  be  sure  there  would  be  a 
sufficient  number  to  carry  out  exhaustive  flight  tests,  and  second,  by  issuing  a  letter  of 
intent  to  contract  for  a  mass-production  order  for  2,500  P-75’s.”  (Holley,  1987:588) 

“Engineering  and  construction  proceeded  at  an  accelerated  rate  following  these  increased 
orders.”  (Thornton,  1946:2) 

“Deliveries  were  expected  to  begin  by  May  1944,  approximately  nine  months  hence,  a 
phenomenally  short  time  for  such  a  complex  undertaking.”  (Holley,  1987:588) 

“When  air  force  [sic]  officers  estimated  that  it  would  take  at  least  six  months  to  eliminate 
these  problems,  pushing  the  projected  date  for  reaching  peak  production  past  the  middle 
of  1945,  it  was  clear  that  the  time  had  come  to  kill  the  whole  project.”  (Holley,  1987:591) 

“One  can  only  speculate  that  it  was  the  urgent  necessity  of  getting  a  superior  fighter  into 
production  as  rapidly  as  possible  that  led  to  the  experiment.”  (Holley,  1987:592) 
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.  the  tremendous  pressure  for  immediate  production  which  did  not  allow  sufficient 
time  for  carefully  planned  development.  In  spite  of  these  conditions,  the  difficulties  were 
largely  corrected  and  a  generally  satisfactory  design  had  evolved  by  the  time  the  first 
production  airplanes  were  ready.”  (Thornton,  1946:2) 


INEXPENSIVE 

“...  the  Materiel  Command  signed  a  cost-plus-fixed-fee  contract  for  two  experimental 
fighters,  XP-75,  at  the  relatively  modest  estimated  cost  of  $428,271,  for  delivery  in  six 
months.”  (Holley,  1987:587) 

“After  considerable  negotiation  the  air  force  [s/c]  agreed  to  $4,450,702  as  the  estimated 
cost  of  the  eight  experimental  airplanes,  but  allowed  GM  a  fixed  fee  of  no  more  than 
$75,163  or  2.65  percent  of  the  original  estimated  total,  a  figure  well  below  the  usual  4  or 
5  percent  fee.”  (Holley,  1987:589) 


SIMPLE 

“Engineering  and  production  was  to  be  expedited  by  using  certain  major  assemblies  such 
as  wing  panels,  empennage,  and  landing  gear  from  existing  production  airplanes  in  a  new 
design  around  the  most  powerful  liquid  cooled  engine  available.”  (Thornton,  1946: 1) 

“By  enlarging  the  tail  surface,  most  of  the  apparent  instability  could  be  overcome,  but 
this  meant  it  would  no  longer  be  possible  to  use  the  ready-built,  off-the-shelf  stock  design 
already  in  production. . .”  (Holley,  1987:590) 

“To  make  matters  worse,  the  Allison  engine,  which  had  been  rushed  into  production 
before  it  was  thoroughly  debugged,  was  not  performing  up  to  expectations. . .”  (Holley, 
1987:590) 

“The  remedy,  eventually  devised,  was  to  extend  the  ailerons  out  to  the  wingtips  to  insure 
greater  stability  and  control.  But  this,  too,  required  reworking  the  standard  ready-built 
wings...”  (Holley,  1987:590) 

“. . .  more  and  more  evidence  emerged  to  indict  the  whole  scheme  to  use  off-the-shelf 
stock  components  to  build  a  superior  high-performance  aircraft.”  (Holley,  1987:591) 


TINY 

“Each  such  modification  added  weight...”  (Holley,  1987:590) 

“Echol’s  espousal  of  the  XP-75  is  all  the  more  curious  in  view  of  the  massive  size  of  the 
plane.  With  its  49-foot  wingspan,  it  was  substantially  heavier  than  virtually  any  other 
successful  World  War  II  fighter.”  (Holley,  1987:592) 
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OUTCOME 

“Every  new  model  aircraft  has  bugs  which  have  to  be  ironed  out;  this  was  to  be  expected. 
But  the  faults  encountered  in  the  XP-75  were  more  than  minor.  The  test  pilot  reported 
that  the  plane  lacked  stability  and  displayed  a  distressing  tendency  to  stall  and  spin  when 
making  tight  turns,  the  maneuver  most  essential  to  fighter  aircraft.”  (Holley,  1987:589) 

“On  October  4,  1944,  the  chief  of  air  staff,  Lt.  Gen.  B.M.  Giles,  signed  the  order 
tenninating  the  contract.”  (Holley,  1987:591) 

ANALYSIS 

The  XP-75  /  P-75  Eagle  project  failed  to  deliver  the  promised  perfonnance,  and 
so  the  project  was  cancelled  shortly  before  World  War  II  ended.  It  thus  earns  a  grade  of  F, 
for  failing  to  provide  a  successful  operational  capability. 

The  XP-75  development  team  clearly  valued  speed.  In  fact,  rapid  development 
was  a  cornerstone  of  the  project.  Along  with  an  aggressively  short  schedule,  the  XP-75 
had  an  austere  budget  and  a  strong  preference  for  using  existing,  mature  technology 
(which  indicates  the  Simple  value).  The  project  leaders  thus  appear  to  have  embraced 
many  of  the  FIST  values.  However,  there  are  a  few  indications  that  the  FIST  approach 
was  not  entirely  adopted  in  this  project,  and  even  when  the  FIST  values  were  present, 
they  were  often  misapplied  or  misunderstood.  Conflicting  values  also  emerged  as  the 
experiment  progressed,  limiting  the  team’s  ability  to  fully  express  the  FIST  values.  When 
World  War  II  ended,  the  need  for  the  P-75  was  gone  and  so  the  project  was  cancelled. 

The  emphasis  on  speed  led  to  cutting  some  comers  in  the  development  planning 
stage  (i.e.  “speeding”).  This  introduced  additional  difficulties  in  the  effort,  which  were 
fortunately  corrected  in  relatively  short  order.  So,  while  the  project  leaders  may  be  guilty 
of  speeding,  they  managed  to  constrain  the  impact  of  that  error. 
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The  real  shortcoming  was  in  the  area  of  complexity  and  simplicity.  Project 
leaders  severely  underestimated  the  complexity  involved  with  integrating  pieces  of 
existing  airframes  into  a  new  airframe.  Each  of  those  pieces  had  been  optimized  for  the 
original  aircraft,  and  most  had  to  be  modified  before  they  could  be  used  on  the  P-75.  The 
designers  seemed  to  have  aimed  for  simplicity  but  instead  achieved  simplisticness. 

Ford  failed  to  appreciate  the  dynamic  nature  of  the  threat  environment,  which 
drives  essential  design  modifications  to  fighter  aircraft.  Holley  points  out  “Changes  in 
design  are  dictated  not  by  the  whim  of  meddling  government  bureaucrats,  as  Ford 
suggested,  but  rather  by  advances  in  the  perfonnance  of  enemy  aircraft  that  must  be 
countered.”  (Holley,  1987:582)  Thornton  similarly  observes  “The  changing  tactical 
situation  soon  resulted  in  modification  of  the  development...”  (Thornton,  1946:1).  While 
the  FIST  approach  tends  to  minimize  requirements  instability,  it  does  not  do  away  with 
instability  altogether,  and  despite  its  aggressive  development  schedule,  the  P-75  was 
exposed  to  more  design  changes  than  many  longer  programs.  This  high  rate  of  instability 
and  necessary  change  limited  the  team’s  ability  to  keep  things  simple. 

Further  complicating  matters,  performance  was  a  top  priority.  Holley  explains 
“...General  Arnold,  in  his  capacity  as  commanding  general  of  the  Army  Air  Forces, 
made  it  clear  that  everything  depended  on  the  performance  of  the  initial  experimental 
aircraft.  ‘If  it  does  not  meet  our  requirements,’  he  wrote,  ‘all  orders  may  be  cancelled; 
everyone  must  understand  this.’”  (Holley,  1987:589)  Expecting  the  initial  experimental 
aircraft  to  perfonn  well  and  meet  all  the  requirements  raises  the  importance  of 
performance  ahead  of  timeline  or  budget,  thus  diminishing  the  role  of  the  FIST  values.  In 
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fact,  the  FIST  approach  by  necessity  has  a  strong  tolerance  for  risk  and  failure,  and  tends 
to  work  best  when  following  an  “imperfectionist”  mindset,  with  the  understanding  that 
rapid  development  allows  for  time  to  try  again.  A  perfectionist  expectation  that  the  initial 
aircraft  flawlessly  meet  all  the  requirements  is  not  compatible  with  the  FIST  approach. 

Clearly,  the  initial  experiment  was  not  a  complete  success.  Holley  writes  “...in 
the  latter  half  of  1942,  the  air  force  was  desperate  to  get  a  superior  fighter...  the  XP-75 
was  a  long  shot...  When  the  experimental  plane  failed  to  achieve  its  promised 
perfonnance,  the  gamble  failed.”  (Holley,  1987:593).  It  is  unknown  whether  subsequent 
attempts  would  have  led  to  better  outcomes.  By  cancelling  the  project  after  a  single 
attempt,  the  Air  Force  reduced  its  opportunity  to  leam  from  the  failure  and  apply  those 
lessons  in  subsequent  trials. 

This  story  also  shows  that  even  a  FIST  project  can  be  overcome  by  events.  The 
end  of  World  War  II  removed  much  of  the  impetus  behind  the  P-75  development,  despite 
the  speed  with  which  it  was  developed.  The  project  was  started  too  close  to  the  end  of  the 
war,  and  once  the  fighting  stopped,  there  was  little  apparent  need  for  an  experimental 
pursuit  aircraft.  The  threat  environment  had  changed,  and  with  it,  the  national  defense 
priorities.  So,  while  FIST  projects  tend  to  minimize  the  risk  of  delivering  capabilities 
after  they  are  no  longer  needed,  it  does  not  guarantee  this  outcome. 
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PROJECT  NAME:  Mine  Resistant  Ambush  Protected  (MRAP)  Vehicles 
DATES:  2007  -  2008 

STORY  SOURCE:  Jones,  2007;  Sullivan,  2008; 

RUBRIC  SCORE:  15 
F- Score:  10 
I-Score:  0 
S-Score:  5 
T -Score:  0 
OUTCOME:  A 
SAMPLE  STATEMENTS: 

FAST 

“In  late  January,  MSCS  [Marine  Corps  Systems  Command]  challenged  industry  by 
awarding  nine  separate  Indefinite  Delivery,  Indefinite  Quantity  (IDIQ)  contracts  for  36 
initial  MRAP  test  vehicles.  The  IDIQ  contracts  required  these  initial  vehicles  to  be 
produced  within  60  days.  However,  in  less  than  three  weeks,  five  of  the  vendors 
demonstrated  their  reliability  to  produce  vehicles  meeting  Marine  Corps  survivability 
requirements,  production  numbers  and  delivery  timelines.”  (Jones,  2007) 

“  ‘We  want  the  maximum  number  of  survivable  vehicles,  with  proven  perfonnance,  in 
the  shortest  time  to  deliver,’”  said  Paul  Mann,  MCSC’s  program  manager  for  MRAP. . . 
‘...we  hope  everyone’s  product  is  as  good  as  they  state  so  we  can  expedite  production 
orders.  Theater  Commanders  have  an  urgent  and  compelling  need  for  these  vehicles.  It  is 
up  to  all  of  us  to  act  fast.’”  (Jones,  2007) 

“...  said  Barry  Dillon,  MSCS’s  executive  director.  ‘The  vehicles...  need  to  be  fielded  as 
soon  as  possible...  We  are  encouraging  [industry]  to  produce  vehicles  faster.  The  faster 
we  can  field  quality,  safe  MRAP  vehicles,  the  more  lives  we  can  save.’”  (Jones,  2007) 

“DOD  designated  the  MRAP  program  as  DOD’s  highest  priority  acquisition,  which 
helped  contractors  and  other  industry  partners  to  more  rapidly  respond  to  the  urgent 
need...”  (Sullivan,  2008:2) 

“The  Secretary  [of  Defense]  also  approved  a  special  designation  for  MRAP — a  DX 
rating — that  requires  related  contracts  to  be  accepted  and  perfonned  on  a  priority  basis 
over  other  contracts  without  this  rating.”  (Sullivan,  2008:9) 


INEXPENSIVE 

“To  date,  more  than  $22  billion  has  been  appropriated  to  acquire  more  than  15,000 
MRAP  vehicles...”  (Sullivan,  2008:1). 

“Although  DOD’s  MRAP  acquisition  approach  has  provided  a  rapid  solution  to 
battlefield  threats,  the  approach  has  also  created  a  number  of  challenges  in  acquiring  and 
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sustaining  mine  resistant  vehicles — challenges  that  will  likely  have  short-  and  long-tenn 
impacts  on  capability  and  budgets.”  (Sullivan,  2008:10) 


SIMPLE 

“DOD  used  a  tailored  acquisition  approach  to  rapidly  acquire  and  field  MRAP  vehicles. 
The  program  established  minimal  operational  requirements  and  relied  heavily  on 
commercially  available  products.”  (Sullivan,  2008:2) 


TINY 

OUTCOME 

“We  ran  over  an  IED  and  the  round  sounded  like  nothing  more  than  an  M-80.  Nobody 
was  injured  except  our  truck  got  a  few  holes  in  the  tool  boxes  and  two  tires  went  flat — 
but  it  was  out  on  another  mission  the  same  day.”  (Jones,  2007) 

“The  highly  concurrent  test  and  production  strategy  for  the  MRAP  vehicle  helped  to 
quickly  identify  the  vehicles  that  met  requirements  for  crew  protection  but  resulted  in  the 
fielding  of  vehicles  with  significant  operational  issues  as  well  as  the  acquisition  of  a 
small  quantity  of  vehicles  that  won’t  be  fielded.”  (Sullivan,  2008:10) 


ANALYSIS 

The  MRAP  team  clearly  emphasized  Fast  above  all,  even  going  so  far  as  to  have 
the  SECDEF  create  a  special  acquisition  category  to  ensure  the  MRAP  moved  forward  as 
fast  as  possible.  As  of  15  Jul  2008,  “about  6,600  of  the  vehicles  have  been  fielded,” 
(Sullivan,  2008:1)  which  is  an  impressive  feat.  It  is  not  entirely  clear,  however,  that  the 
MRAP  team  avoided  some  of  the  “speeding”  pitfalls,  such  as  increased  costs  and 
schedule  delays.  As  Sullivan  points  out,  “DOD’s  plan  for  logistical  support  beyond  the 
first  2  years  is  still  being  developed.”  (Sullivan  2008:11).  This  might  be  evidence  of  an 
avoidable  strategic  cost  caused  by  tactical,  short-term  thinking. 

MRAP  leaders  emphasized  speed  and  sacrificed  cost,  potentially  driving  up  costs 
to  a  greater  degree  than  necessary.  One  wonders  what  would  have  happened  if  the  team 
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had  implemented  the  Inexpensive  value,  alongside  the  Fast  value,  and  insisted  on  being 
both  Fast  and  Inexpensive.  However,  some  of  the  decisions  made  in  the  interest  of  speed 
have  the  potential  to  help  keep  costs  down.  For  example,  using  9  different  IDIQ  contracts 
could  foster  competition  between  vendors,  which  is  precisely  the  kind  of  thing  a  project 
leader  would  do  to  express  the  Inexpensive  value.  Unfortunately,  the  vendors  are  not 
working  from  a  common  blueprint  and  each  type  of  MRAP  is  different  from  the  others, 
making  maintenance  and  operations  slightly  more  complicated. 

To  date,  the  operational  outcome  seems  positive.  Marines  and  soldiers  who  use 
the  vehicles  seem  genuinely  satisfied  and  reports  indicate  many  lives  have  been  saved. 
However,  it  is  still  too  early  to  say  definitively  whether  the  MRAP  will  be  a  success  over 
the  long  run. 
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PROJECT  NAME:  E-3  Sentry  (AWACS) 

DATES:  1965  -  1977 

STORY  SOURCE:  Grier,  2002;  Cowdery  &  Skillman,  1995;  GAO  report,  1973; 
Ulsamer,  1974; 

RUBRIC  SCORE:  25 

F -Score. TO 
I- Score:  5 
S-Score: 10 
T -Score: 0 

OUTCOME:  A 
SAMPLE  STATEMENTS: 

FAST 

On  Dec  22,  1965 . . .  the  AWACS  effort  was  officially  born. . .  in  March  1977,  the  first 
AWACS  was  formally  delivered  to  Tactical  Air  Command. . (Grier,  2002:2,3) 

“The  Brassboard  radars  were  designed,  fabricated,  tested  and  installed  in  the  flight  test 
aircraft  at  Boeing  in  less  than  two  years. . .  This  was  rather  incredible  considering  the 
complexity  of  the  radar  and  instrumentation.”  (Cowdery  &  Skillman,  1995:5) 

“Because  of  the  ‘crash’  nature  of  the  program,  most  of  the  system  debugging  occurred  at 
Boeing.”  (Cowdery  &  Skillman,  1995:5) 

“A  crash  effort  resulted  in  a  redesigned  master  oscillator  that  provided  adequate  stability 
and  was  installed  soon  after  flights  began.”  (Cowdery  &  Skillman,  1995:6) 

“On  November  7,  1972,  Boeing  successfully  completed  the  Airborne  Tracking 
Demonstration,  4  months  ahead  of  schedule.”  (GAO,  1973:1) 

“In  the  view  of  AWACS  System  Program  Director  Brig.  Gen.  Lawrence  A.  Skantze,  a 
major  delay  in  the  program  might  be  tantamount  to  its  termination.”  (Ulsamer,  1974:70) 

“The  significant  schedule  dates  in  the  September  1972  SAR  remained  unchanged  except 
for  the  “Rollout  Of  The  First  DDT&E  Aircraft”  milestone  which  improved  by  4  months  - 
-  from  May  1975  to  January  1975.  This  improvement  resulted  from  use  and 
reconfiguration  of  one  of  the  two  Brassboard  aircraft  —  the  Hughes  radar  equipped 
aircraft  -  instead  of  waiting  to  develop  and  equip  a  third  aircraft.”  (GAO,  1973:5) 

“Brig.  Gen.  Lawrence  A.  Skantze,  AWACS  System  Program  Director,  stresses  that  the 
E-3  A  development  and  acquisition  effort  ‘has  been  tailored  to  proceed  at  a  deliberate 
pace,  leaving  time  for  review  at  distinct  phases  of  the  program. . .”  (Ulsamer,  1974:71) 

“According  to  General  Skantze,  the  AWACS  test  program  focused  on  two  key  objectives 
from  the  very  beginning:  To  identify  the  most  difficult  task  and  to  ensure  that  it  is  being 
tested  early  and,  secondly,  to  provide  a  structured  test  program  that  will  provide,  at  each 
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milestone,  solid  evidence  of  the  performance  status  of  the  system  and  its  subelements.” 
(Ulsamer,  1974:76). 

“Because  of  the  success  of  the  radar  test  program,  it  became  possible  to  speed  up  the 
AWACS  test  and  evaluation  schedule  across  the  board.”  (Ulsamer,  1974:76). 


INEXPENSIVE 

“. . .  in  January  1973,  the  approved  program  was  based  on  a  4  engine  rather  than  an  8 
engine  configuration;  had  a  program  cost  of  $2,467  million,  for  a  total  reduction  of  $194 
million...”  (GAO,  1973:  2) 

“As  of  January  19,  1973,  the  approved  program  cost  estimate  was  reduced  to  $2,467 
million.  The  Air  Force  is  to  make  more  cost  reduction  . . .  program,  a  cost  goal  of  $2,284 
million  having  been  established.”  (GAO,  1973:3  -  copy  damaged  and  partially 
unreadable). 

“Budgetary  constraints  at  the  time  of  AWACS’s  first  DSARC  in  July  1970  caused  the  Air 
Force  to  ‘trade  down’  to  a  basic  AWACS  system...”  (Ulsamer,  1974:76). 

“. . .  some  reduction  in  time-on-station  was  more  than  made  up  for  by  the  lower  cost  of 
using  TF33  engines  already  in  the  Air  Force’s  inventory  in  surplus  quantities.”  (Ulsamer, 
1974:76). 

SIMPLE 

“. . .  the  Pentagon  treated  development  of  the  system  as  a  high-priority  effort.  For  example, 
AWACS  had  its  own  streamlined  procurement  rules...”  (Grier,  2002:2) 

‘“The  total  complexity  of  this  system. . .  far  exceeds  things  I’ve  worked  on  before,’  said 
Ed  Froesse,  Boeing’s  vice  president  for  the  AWACS  program.”  (Grier,  2002:5) 

“To  deal  with  the  many  problems  that  arose  during  the  flight  tests,  both  contractors 
shipped  many  of  the  engineers  to  Boeing  so  they  would  be  available  for  trouble-shooting 
around-the-clock.”  (Cowdery  &  Skillman,  1995:5) 

“There  is  no  compelling  reason  for  lengthy  airframe  and  engine  testing  because  the 
AWACS  aircraft  is  a  modified  Boeing  707-320. . .  AWACS  engines  are  those  of  the  Air 
Force’s  C-141,  similarly  proved  in  many  years  of  service.”  (Ulsamer,  1974:70) 


TINY 


OUTCOME 

“Twenty- five  years  ago,  on  March  24,  1977,  Boeing  delivered  the  first  basic  production 
version  of  the  E-3  Sentry  to  Air  Force  officials  at  Tinker  AFB,  Okla.  The  ensuing  quarter 
century  has  shown  the  AWACS  to  be  indispensible,  often  the  first  system  to  go  into 
action  when  a  threat  arises  and  the  last  to  leave  once  operations  cease.”  (Grier,  2002: 1) 
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ANALYSIS 

The  E-3  Sentry  (AWACS)  development  effort  is  an  excellent  example  of  the 
importance  of  context  when  assessing  a  project.  The  Air  Force  spent  a  lot  of  money 
during  the  1 1  years  it  spent  developing  the  AWACS,  and  the  aircraft’s  avionics  was  often 
described  as  highly  complex.  At  first  glance,  this  seems  to  be  the  opposite  of  the  FIST 
values.  However,  the  unique  nature  of  the  aircraft  leaves  analysts  with  few  (if  any) 
appropriate  systems  for  comparison.  Upon  closer  inspection,  there  are  many  indications 
that  the  FIST  values  guided  and  shaped  the  decisions  made  by  AWACS  project  leaders. 

More  than  1 1  years  passed  between  the  start  of  the  AWACS  effort  and  delivery  of 
the  first  operational  aircraft.  However,  within  that  timeframe,  project  leaders  seemed  to 
place  great  value  on  speed,  even  stating  that  a  significant  delay  would  be  cause  for 
tennination.  BGen  Skantze  was  essentially  saying  that  it  was  important  and  good  to 
deliver  on  time,  and  slow  delivery  might  as  well  be  no  delivery,  thus  providing  evidence 
of  the  Fast  value.  The  developers  made  a  point  of  focusing  on  the  difficult  projects  early, 
rigorously  tested  components,  and  cleverly  avoided  the  temptation  to  cut  comers  in  the 
name  of  speed.  Recall  that  a  system’s  F-score  is  not  based  on  how  long  it  takes  to  deliver, 
but  on  whether  speed  is  valued. 

Early  critics  complained  the  AWACS  cost  too  much  and  would  not  perform  well. 
The  critics  were  clearly  wrong  about  the  performance,  but  their  assessment  of  the  cost  is 
worth  considering.  Project  leaders  were  willing  to  slightly  reduce  performance  (time-on- 
station)  in  order  to  save  money  by  using  existing  engines.  Had  they  valued  perfonnance 
more  than  cost,  they  would  have  pushed  to  spend  more  money  on  new  engine  designs.  In 
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1973  (four  years  before  the  first  delivery)  the  GAO  reported  some  cost  reductions,  but 
those  reductions  correlated  with  a  decrease  in  the  number  of  airframes,  from  5  down  to  3 
DT&E  aircraft.  (GAO,  1973:2).  This  is  not  necessarily  a  direct  sign  of  the  Inexpensive 
value,  but  in  the  overall  context,  it  seems  to  indicate  the  value’s  presence. 

The  AWACS  is  physically  very  complex,  with  an  impressive  number  of 
interconnected  components,  functions,  sub-systems  and  missions.  With  a  flight  crew  of 
four  and  a  mission  crew  of  13-19,  it  also  has  one  of  the  largest  crew  size  of  any  military 
aircraft,  contributing  further  to  its  complexity.  However,  project  leaders  simplified  as 
often  as  they  could.  Specifically,  they  used  an  existing  airframe  and  existing  engines, 
incorporating  mature  technology  whenever  possible.  Even  the  procurement  rules  were 
streamlined. 


The  E-3  Sentry  is  a  widely  recognized  for  making  significant  contributions  to 
national  defense,  and  has  a  long,  proud  history  of  operational  success,  thus  earning  an 
operational  grade  of  A. 

CO-EVALUATOR  ANALYSIS 

F:  High  (10)  (several  instances  of  wanting  to  move  program  quickly  but  still  smartly) 

I:  High  (10)  (some  discussion  on  reducing  budget  and  cutting  back  number  of  engines 
and  total  number  of  aircraft  as  well  as  organizing  to  provide  for  future  block  upgrades 
down  the  road) 

S:  Med  (5)  (little  on  the  fence  between  H/M  since  built  on  previous  and  avail  tech  but  did 
require  pretty  good  amount  of  additional  R&D  and  test  so  went  med) 

T:  Low  )0)  (didn’t  really  see  much  in  this  area  except  mentions  of  expanding  to  include 
NATO/allies) 

Outcome:  A 
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PROJECT  NAME:  NASA’s  Viking 
DATES:  1970  -  1980 

STORY  SOURCE:  McCurdy,  2001 
RUBRIC  SCORE:  -10 

F-Score:  0 
I-Score:  -5 
S-Score:  -5 
T -Score:  0 

OUTCOME:  A 

SAMPLE  STATEMENTS: 

FAST 

“The  Pathfinder  team,  moreover,  was  asked  to  design,  build,  and  prepare  their  spacecraft 
for  launch  in  just  3  years.  The  Viking  team  had  taken  6.”  (61) 


INEXPENSIVE 

“Viking  was  a  classic  NASA  mission:  complex,  redundant  and  expensive.”  (61) 

“The  final  cost  of  the  Viking  biology  instrument,  however,  was  much  higher.  Due  to  cost 
overruns,  the  biology  instrument  actually  cost  $220  million  (inflation-adjusted)  dollars — 
5  times  the  original  estimate.”  (68) 

“When  the  project  began,  Viking  managers  estimated  that  the  imaging  system  could  be 
designed  and  built  for  $6.2  million.  The  actual  cost  ($27.3  million)  was  4  times  that 
amount.”  (70) 

“NASA  officials  originally  estimated  that  they  could  develop  the  biology  package  for 
$1 1.3  million.  The  actual  cost  was  $59.5  million.”  (72) 

“[the  Viking’s  computer’s]  cost  grew  from  $3.4  to  $28.1  million.”  (74) 

“The  Viking  team  spent  $5 1  million  on  initial  operations  in  the  period  immediately 
following  the  first  landing.  In  inflation-adjusted  dollars,  that  is  $174  million  more  than 
spent  by  the  Pathfinder  team.” 


SIMPLE 

“Viking  team  members  designed  an  elaborate  landing  system,  with  computers,  parachutes, 
radar  altimeters,  and  a  throttleable  rocket  engine.  They  developed  a  redundant  radio 
communication  system. . .”  (65) 

“The  Viking  landers  had  more  capability  than  the  Pathfinder  spacecraft.”  (68) 
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“Scientists  recommended  that  project  engineers  incorporate  a  simpler  design  with  less 
capability...  The  technically  sophisticated  design  was  retained.”  (71) 

“The  biology  package  weighed  33  pounds  and  contained  40,000  parts.”  (72) 

“The  only  way  to  pack  four  experiments  into  a  small,  lightweight  container  was  to  make 
the  package  complex.  In  an  instrument  this  complex,  any  single-point  failure  threatened 
the  whole  system.”  (72) 


TINY 

“The  Viking  mission,  by  contrast  [with  Pathfinder ]  required  a  larger  launch  vehicle.”  (64) 

“The  Viking  cameras  were  bulky  affairs. . .  Each  lander  had  two.”  (70) 

“The  people  developing  the  biology  package  were  under  tremendous  pressure  to 
simultaneously  reduce  cost,  weight,  and  complexity.”  (72) 

“The  computer  on  the  Viking  lander  weighed  52  pounds. . .  The  Viking  lander  required  17 
times  as  much  electric  power  as  the  Pathfinder  lander.”  (74) 

“The  Viking  project,  at  the  height  of  its  activity,  employed  538  people  at  the  Langley 
Research  Center,  1,650  people  at  the  Martin  Marietta  Aerospace  Center,  an  unknown 
number  of  people  at  the  Jet  Propulsion  Laboratory,  and  69  scientists  on  13  advisory 
teams.”  (75) 


OUTCOME 

“Between  1962  and  1996,  the  United  States  and  the  Soviet  Union  sent  19  robotic 
missions  to  Mars.  Only  six  successfully  completed  their  objectives. . .  Mars,  it  would 
seem,  eats  spacecraft. . .  The  Viking  program  bought  two  spacecraft  that  landed  on  the 
Martian  soil  and  two  spacecraft  that  studied  the  planet  from  orbits  around  Mars.”  (61) 

“Although  the  resulting  images  were  spectacular,  Viking  program  managers  paid  for  this 
decision  with  substantial  cost  overruns.”  (71) 


ANALYSIS 

There  is  no  evidence  the  Viking  project  team  held  to  the  LIST  values.  The 
program  was  consistently  described  as  complex,  redundant  and  expensive  in  several 
sources  quoted  above,  by  advocates  as  well  as  critics.  Performance  and  reliability  were 
clearly  more  important  to  project  leaders  than  speed,  simplicity  or  low-cost  technologies. 
It  was  not  considered  a  negative  that  the  project  cost  so  much,  took  so  long  or  was  so 
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large  and  expensive.  In  fact,  when  simpler  designs  were  proposed,  they  were  rejected  in 
favor  of  more  complex  solutions.  The  Viking  team  was  committed  to  taking  as  much 
time  as  necessary  to  ensure  optimal  performance  and  minimize  risks  on  this  very  difficult 
mission. 

The  program  was  clearly  successful,  as  Viking  operated  for  years  in  one  of  the 
most  difficult  environments  NASA  has  ever  attempted.  However,  it  is  worth  mentioning 
that  the  effort  was  not  replicated  and  the  US  did  not  attempt  a  second  Mars  landing  until 
Pathfinder,  20  years  later. 
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PROJECT  NAME:  C-5A  Galaxy 
DATES:  1957  -  1973 

STORY  SOURCE:  Gregory,  1989;  AF  Center  for  Systems  Engineering  C-5A  Case 
Study,  2005; 

RUBRIC  SCORE:  -15 

F-Score:  0 
I -Score:  -5 
S-  Score:  -5 
T- Score:  -5 

OUTCOME:  A 
SAMPLE  STATEMENTS: 

FAST 

“It  resulted  in  the  publication  of  the  MATS  Qualitative  Operational  Requirements  (QOR) 
in  October  1961  [1]  and  the  System  Operational  Requirements  (SOR)  214  for  Heavy  Lift 
Logistics  Weapon  System  on  25  March  1964,  and  the  release  of  the  RFP  in  December 
1965.  Note  that  the  whole  process  took  seven  years,  starting  with  the  early  vision,  to 
build  the  program  consensus,  develop  the  budget  plan,  and  define  the  requirements.” 
(CSE  Case  Study,  2005:16) 

“The  first  flight  of  the  C-5A  occurred  on  28  June  1968,  33  months  after  contract  award” 
(CSE  Case  Study,  2005:14) 

“Operational  effectiveness/mission  analysis  studies  covered  range,  payload,  speed,  and 
airport  conditions,  both  at  takeoff  and  landing.  The  entire  cargo/transport  aircraft 
community,  the  contractors,  the  development  planners,  the  users,  and  the  technologists 
were  deeply  involved  in  the  assessment  process.  It  resulted  in  the  publication  of  the 
MATS  Qualitative  Operational  Requirements  (QOR)  in  October  1961  [1]  and  the  System 
Operational  Requirements  (SOR)  214  for  Heavy  Lift  Logistics  Weapon  System  on  25 
March  1964,  and  the  release  of  the  RFP  in  December  1965.  Note  that  the  whole  process 
took  seven  years,  starting  with  the  early  vision,  to  build  the  program  consensus,  develop 
the  budget  plan,  and  define  the  requirements.”  (CSE  Case  Study,  2005:16) 

“The  C-5A  Program  was  characterized  by  an  excellent  pre-Milestone  A  and  Milestone  A- 
to-Milestone  B  process.”  (AF  Studies  Board,  2008:37) 

“The  time  available  to  redesign  the  wing  was  inadequate  to  conduct  a  systems 
engineering  assessment  of  the  new  design. . .  The  net  effect  of  all  this  activity  during 
source  selection  was  to  excessively  constrain  costs,  schedule  and  performance  to  such 
tight  and  rigorous  limits  that  there  was  no  hope  of  being  able  to  perform  to  the 
requirements  of  the  contract.”  (AF  Studies  Board,  2008:37) 
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“The  Air  Force  spent  over  $2  billion  more  than  the  original  budget,  and  it  took  10  years 
longer  to  finish  the  production  with  fewer  aircraft  than  originally  planned.”  (AF  Studies 
Board,  2008:38) 


INEXPENSIVE 

“As  for  the  C-5A,  it  was  a  watershed.  Its  cost  overrun  crossed  the  billion-dollar  line,  thus 
engineering  the  transition  of  public  perception  of  the  military-buying  system  and  the 
defense  industry  from  a  team  that  could  invent  anything,  that  could  turn  technology  into 
first-rate  hardware  unequalled  anywhere,  to  an  apparatus  that  was  wasteful,  dishonest  and 
incapable  of  producing  anything  that  worked  or  stayed  within  cost  targets.”  (Gregory, 
1989:107-108) 

“C-5A  program  was  plagued  by  cost  overruns  from  the  very  beginning.  The  cost  overrun 
was  not  unexpected  by  either  the  Air  Force  or  the  contractor.  The  Air  Force  had  predicted 
a  most  probable  cost  of  $  1 .86  billion  as  opposed  to  the  contractor’s  best  and  final 
estimate  of  $1.71  billion  on  4  September  1965.  The  actual  cost  of  the  original  contract 
grew  by  another  $64  million  by  December  1965.  No  one  estimated  costs  as  high  as  the 
eventual  program  overrun  to  $3.5  billion  for  the  development  effort  plus  the  115  aircraft 
in  Run  A  and  Run  B.”  (CSE  Case  Study,  2005:22) 

“The  enormous  cost  overrun  had  multiple  causes.  Key  reasons  included:  (1)  the  original 
cost  estimate  was  aggressively  low,  influenced  by  the  contractor’s  expectation  of  an 
ancillary  benefit  in  the  form  of  follow-on  for  commercial  air  vehicle  design;  (2)  the 
perceived  low  cost  estimates  of  the  competitors...  ”(CSE  Case  Study,  2005:22) 


SIMPLE 

“. . .  the  C-5  was  to  have  an  unusual  drive-through  main  cabin,  with  vast  cargo  doors  at 
both  front  and  rear.  In  fact,  the  whole  nose  section  of  the  airplane  swung  up  and  over  the 
cockpit.  And  the  landing  gear  could  kneel,  adjusting  on  the  ground  for  differing  loading- 
ramp  heights.  On  top  of  that  complexity,  the  C-5A  looked  like  a  winged  centipede  on  the 
ground  because  of  short  but  heavy  struts  supporting  a  couple  of  dozen  balloon  tires  to 
allow  it  to  land  on  grass  fields.”  (Gregory,  1989:109) 


TINY 

“But  building  a  big  airplane  like  the  C-5  A  was  not  just  a  matter  of  scaling  up  the  smaller 
C-141.  When  the  increase  in  size  passed  a  certain  point,  technical  challenges  and  changes 
cropped  up.”  (Gregory,  1989: 1 10) 

“When  the  C-5  aircraft  first  entered  the  inventory  in  1970,  it  was  the  largest  aircraft  in  the 
world,  and  it  is  still  the  largest  transport  cargo  aircraft  in  our  inventory.  It  was  the  first  of 
the  behemoths.”  (CSE  Case  Study,  2005 :v) 
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OUTCOME 


“The  success  of  the  C-5  transport  aircraft  is  underscored  by  the  perfonnance  of  the  fleet 
as  an  operational  system  and  its  heavy  lift  support  in  all  of  our  conflicts  from  Vietnam  to 
Iraq.  It  still  accomplishes  tasks  that  no  other  military  aircraft,  such  as  the  new  C-17  or 
any  derivative  of  commercial  cargo  aircraft,  can  perfonn,  and  has  consistently  carried 
more  cargo  than  any  other  aircraft  in  the  time  of  war.”  (CSE  Case  Study,  2005  :vi) 

“The  C-5A  fleet  has  demonstrated  success  in  operation,  but,  as  will  be  discussed,  the 
development  of  the  aircraft  was  plagued  by  a  major  technical  failure  in  the  design  of  the 
structure,  most  notably  the  wing  and  pylon.”  (CSE  Case  Study,  2005:5-6) 


ANALYSIS 

The  C-5A  Galaxy  is  the  largest  aircraft  in  the  US  inventory,  with  a  successful 
operational  track  record  that  extends  over  four  decades.  It  provides  unique  capabilities  to 
the  military  and  is  clearly  an  operational  success.  While  some  statements  from  the  C-5A 
stories  may  initially  seem  to  be  value  clues  in  support  of  the  FIST  values,  closer 
examination  reveals  this  to  not  be  the  case.  The  development  effort  cost  $2B  more  than 
originally  envisioned,  and  it  was  delivered  10  years  behind  schedule. 

These  unfortunate  programmatic  outcomes  were  largely  driven  by  the  complexity 
and  size  of  the  aircraft,  and  the  project  leaders’  emphasis  on  delivering  a  new  system 
using  unproven  capabilities  and  technologies.  This  preference  for  complexity  and 
immature  technologies  lead  to  extensive  delays  and  increased  costs. 

While  technical  problems  had  a  negative  impact  on  cost  containment,  the  cost 
problems  can  largely  be  traced  to  the  fact  that  neither  the  government  nor  the  contractors 
believed  (or  intended  to  achieve)  the  initial  cost  estimates.  That  is,  there  was  an  infonnal 
mutual  agreement  that  the  aircraft  would  not  keep  to  the  aggressive  budget,  despite  the 
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implementation  of  a  new  methodology  called  Total  Package  Procurement  Concept 
(TPPC). 

Secretary  of  Defense  McNamara  described  TPPC  as  a  way  to  “scientifically 
manage  a  major  weapon  system”  (CSE  Case  Study,  2005:22),  but  it  ended  up 
contributing  to,  rather  than  preventing,  cost  and  schedule  over  runs.  This  pattern  is 
consistent  with  many  acquisition  reform  efforts,  which  inevitably  end  up  exacerbating  the 
problems  they  are  designed  to  solve  (Ward,  2009). 

It  worked  like  this:  with  the  government’s  full  knowledge  and  tacit  approval,  the 
contractor  deliberately  underbid,  based  partly  on  a  perception  that  its  competitors  were 
also  submitting  low  bids.  Rather  than  viewing  cost  and  schedule  constraints  as  positive 
and  valuable,  they  expected  to  make  up  the  difference  on  the  subsequent  contract  phases, 
in  which  they  hoped  to  secure  additional  funds  and  time.  Indeed,  they  eventually  got  an 
extra  $2B  and  10  years.  These  overruns  were  higher  than  expected,  but  the  fact  that  they 
happened  in  the  first  place  was  a  surprise  to  no  one. 

This  situation  highlights  the  difference  between  project  leaders  who  actually 
understand  and  value  the  FIST  values  and  those  who  are  merely  paying  lip  service  to  the 
idea  of  being  fast,  inexpensive,  simple  and  tiny.  The  TPPC-induced  constraints  were 
viewed  as  barriers  to  be  overcome,  not  as  goals  to  achieve.  Speed,  low-cost  and 
simplicity  were  not  measures  of  merit,  nor  were  they  considered  desirable  attributes.  The 
end  result  was  that  the  USAF  spent  $2B  more  than  expected  and  waited  an  extra  decade 
for  delivery  of  the  capability. 
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PROJECT  NAME:  Crusader  Artillery 
DATES:  1995  -2002 

STORY  SOURCES:  Tracy,  2004;  Loeb,  2002;  Emerson,  2002;  Erwin,  2003;  Veith, 
2002;  Rumsfeld,  2002;  Charette,  2008. 

RUBRIC  SCORE:  -15 

F-Score:  -5 
I-Score:  -  5 
S-Score:  0 
T-Score:  -5 

OUTCOME:  F  -  Program  Cancelled 
SAMPLE  STATEMENTS: 

FAST 

.  .in  development  since  1994. . .  The  anny  wanted  to  buy  480  crusaders,  the  first  of 
which  would  go  into  service  in  2008.”  (Loeb,  2008). 

“Crusader  is  fully  funded,  on  schedule  and  within  budget;  it  will  start  fielding  to  the 
Counterattack  Corps  in  2008.”  (Emerson,  2002)  [NOTE:  Crusader  was  cancelled  two 
months  after  Emerson’s  article  was  published] 


INEXPENSIVE 

“Defense  Secretary  Donald  H.  Rumsfeld  announced  yesterday  he  intends  to  cancel  the 
Anny’s  $11  billion  Crusader  artillery  program  so  that  money  could  be  invested  in  more 
futuristic  weapons  technologies.”  (Loeb,  2002) 

“. . .  the  debate  over  Crusader  is  about  whether  to  spend  roughly  $9  billion  more  to 
procure  some  480  Crusader  howitzers  or,  instead,  use  funds  to  accelerate  a  variety  of 
precision  munitions. . .”  (Rumsfeld,  2002: 10). 


SIMPLE 

“Crusader  will  use  a  turbine  engine  that  will  be  common  with  the  M1A2  system 
enhancement  program  (SEP)  tank.”  (Emerson,  2002:43) 

“. . .  we  have. . .  very  little  expertise  in  some  of  the  more  unproved  aspects  of  the  Crusader. 
For  example,  the  system  is  designed  to  be  heavily  automated,  but  automated  systems  fail 
and  the  manual  back-ups  we  would  need  pose  [sic]  are  a  challenging  dimension  that  is 
relatively  immature  and  unproven.”  (Rumsfeld,  2002:12) 

“The  design  for  the  Army’s  Crusader  howitzer,  for  instance,  relied  on  16  ‘critical’ 
technologies. . .  but  only  six  of  those  technologies  had  ever  been  demonstrated  outside  the 
laboratory  when  the  Crusader  entered  development  in  1994.”  (Charette,  2008:38) 
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TINY 


“The  howitzer’s  many  drawbacks  included — being  too  heavy  for  rapid  deployment.” 
(Tracy,  2004:32) 

“If  Crusader  had  been  much  lighter,  it  could  have  been  the  ‘first  vehicle  of  the  Objective 
Force,’  said  Army  Gen.  Kevin  P.  Byrnes,  head  of  the  Training  and  Doctrine  Command... 
The  Anny  stuck  with  Crusader  for  many  years,  despite  the  weight  problems.”  (Erwin, 
2003:1) 

“Besides  its  inability  to  provide  precise  targeting,  the  Crusader  is  extremely  difficult  to 
deploy,  Rumsfeld  said.  Each  mobile  cannon  weighs  about  40  tons,  and  must  be 
accompanied  by  a  resupply  vehicle  weighing  34  tons.”  (Veith,  2002:A-20). 

“While  it  is  true  that  Crusader  will  not  fit  on  a  C-130,  the  aircraft  used  for  strategic  airlift 
are  the  C-17  and  the  C-5  transports.  Despite  the  Objective  Force  requirement  to  be 
transportable  on  a  C-130,  the  strategic  aircraft  are  the  ones  most  likely  to  be  used  in  an 
initial  strategic  deployment.”  (Emerson,  2002:45) 

“Operations  will  be  conducted  by  a  section  that  is  a  one-third  smaller  than  today’s 
Paladin  section.”  (Emerson,  2002:44) 


OUTCOME 

“The  howitzer’s  many  drawbacks  included — . . .  Being  too  much  of  an  area  fire  weapon 
and  lacking  precision  engagement.”  (Tracy,  2004:32) 

“. .  .to  clear  the  defense  budget  of  what  he  [Rumsfeld]  sees  as  Cold  War  relics. . .”  (Loeb, 

2002) 

“. . .  it  is  about  foregoing  a  system  originally  designed  for  a  different  strategic  context,  to 
make  room  for  more  promising  technologies.”  (Rumsfeld,  2002:9). 


ANALYSIS 

Two  months  before  Crusader  was  cancelled,  advocates  claimed  the  project  was  on 
schedule,  on  budget  and  a  mere  6  years  away  from  being  fielded.  However,  there  was  no 
evidence  the  project  team  valued  short  timelines,  small  budgets  or  simplicity,  much  less 
being  small.  On  the  contrary,  the  project  leaders  planned  to  spend  a  total  of  14  years 
developing  the  large,  complex  system,  for  a  cost  of  $1  IB.  After  more  than  seven  years  in 
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development,  delivery  was  still  over  six  years  away,  which  indicates  that  speed  was  not 
exactly  a  core  value  for  the  project  leaders. 

The  fact  that  Crusader  was  too  large  to  be  transported  on  a  C-130  was  dismissed 
as  trivial,  since  the  larger  C-5  and  C-17  aircraft  could  provide  strategic  transportation. 
This  willful  defiance  of  the  actual  operational  requirements  (and  rejection  of  the  Tiny 
value)  was  a  major  contributor  to  the  program’s  cancellation. 

The  outcome  was  clearly  a  failure  on  every  level.  Project  leaders  spent  $2B  over 
eight  years,  trying  to  produce  a  system  that  ultimately  did  not  meet  the  warfighter’s  needs, 
present  or  projected.  Crusader’s  capabilities  simply  did  not  match  with  the  required 
functions  in  a  post-Cold  War  threat  environment,  so  the  additional  $9B  planned  to  be 
spent  on  the  program  could  not  be  justified.  During  the  time  between  program  start  and 
2002,  the  external  threat  had  changed  to  such  a  degree  that,  even  if  Crusader  had  been 
somewhat  smaller,  cheaper  and  more  easily  deployed,  it  would  have  still  proven 
unnecessary.  Thus,  the  project  was  cancelled  to  free  up  billions  of  dollars  for  more 
pressing  combat  needs,  without  a  single  howitzer  being  delivered. 
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PROJECT  NAME:  F-15  Eagle 
DATES:  1968  -  1976 

STORY  SOURCE:  Hehs,  1991;  Tobias  and  others,  1982;  Burton,  1993;  Air  Force  Fact 
Sheet;  Coates  &  Neikirk,  1981 

RUBRIC  SCORE:  -20 

F-Score:  -5 
I-Score:  -5 
S-Score:  -5 
T-Score:  -5 

OUTCOME:  A 
SAMPLE  STATEMENTS: 

FAST 

Apr  1965:  HQAF  initiated  F-X  program,  calling  for  IOC  in  early  1970’s 
Sep  1975:  First  F-15  squadron  reached  IOC 

INEXPENSIVE 

“The  F-15  was  very  expensive.  It  had  been  twenty-five  years  since  the  Air  Force  had  had 
an  air-superiority  fighter.  It  had  taken  more  than  five  years  just  to  get  the  program 
approved.  They  couldn’t  afford  to  take  any  risks.”  (Hehs,  1991) 


SIMPLE 

“The  weapons  and  flight  control  systems  are  designed  so  one  person  can  safely  and 
effectively  perform  air-to-air  combat.”  (Air  Force  F-15  Fact  Sheet) 

“He  [Chuck  Spinney]  pointed  out  that  the  high-tech  systems  designed  to  make  these 
airplanes  more  reliable  and  cheaper  to  maintain  than  the  older  airplanes  in  the  inventory 
had  an  exactly  opposite  effect.  The  systems  failed  more  often  than  predicted,  and  the  time 
and  costs  to  repair  the  failures  were  far  greater  than  predicted.”  (Burton,  1993:74) 


TINY 

“The  F-15  was  viewed  as  highly  sophisticated  because  it  is  so  big  and  expensive.”  (Hehs, 
1991) 

“. . .  the  F-15  went  through  several  modification  to  beef  up  its  wings  and  add  more  fuel  so 
it  could  pull  as  many  g’s  and  fly  as  far  as  its  little  brother  [the  F- 1 6] .  The  added  weight 
further  increased  its  maneuvering  inferiority  to  the  F-16.”  (Burton,  1993:21) 
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OUTCOME 


. .  a  major  problem  with  big,  complex  fighters  is  their  cost.  A  large  F-15  can  usually 
beat  a  smaller  F-5  in  a  one-on-one  dogfight.  But  we  could  buy  five  F-5’s  for  the  price  of 
one  F-15.  No  one  claims  that  an  F-15  could  beat  five  F-5’s  at  once.  In  fact,  the  Nevada 
exercises  demonstrated  that  the  technical  advantages  one  plane  might  have  over  another 
disappear  quickly  when  more  than  two  planes  are  involved.”  (Tobias  and  others, 
1982:367) 

“[the  F-5]  embarrassed  the  Air  Force  and  Navy  in  the  mock  combat  tests  in  1977,  when  it 
fought  the  vaunted  F-15  and  F-14  to  a  virtual  draw.”  (Burton,  1993:99) 

“The  proud  ‘air  superiority  fighters,’  F-15s  and  F-14s  costing  upward  of  $30  million 
apiece,  had  been  fought  to  all  but  a  draw  by  a  comparatively  crude  $4  million  airplane, 
the  F-5  that  Northrop  Corp.  builds  for  export  to  small  countries.”  (Coates  &  Neikirk, 
1981:1) 

“One  on  one,  the  F-5  was  no  match  for  the  big  planes,”  he  [Col  Robert  Fay,  USAF,  ret] 
said.  “The  14s  and  the  15s  could  score  a  kill  nearly  all  the  time.  But  as  the  numbers  went 
up,  such  as  two  F-5s  against  one  F-15  or  four  F-5’s  against  four  F-15s,  the  picture 
changed  dramatically,”  said  Fay.  “Suddenly  all  the  magic  weapons  were  failing.”  Other 
analysts. . .  have  concluded  that  the  complex  radars  and  other  systems  in  the  big  planes 
became  confused  as  the  number  of  combatants  increased.”  (Coates  &  Neikirk,  1981:12) 

“The  plane  has  been  plagued  with  all  sorts  of  engine  problems,  the  most  disconcerting  of 
which  has  been  a  tendency  to  stall  on  takeoff.”  (Tobias  and  others,  1982:369) 


ANALYSIS 

The  F-15  is  consistently  described  using  words  like  big,  complex  and  expensive — 
and  not  only  by  its  critics.  These  labels  are  often  used  with  pride,  as  signs  of  the  Eagle’s 
sophistication  and  importance.  These  are  interesting  clues  about  the  values  which  drove 
its  development.  The  F-15’s  project  leaders’  preference  for  “high-technology”  and  their 
failure  to  focus  on  simplicity  led  to  several  increases  in  cost  and  size,  and  by  some 
accounts  degraded  its  performance. 

The  Eagle’s  price  tag  reduced  the  project  leaders’  risk  tolerance,  forcing  them  to 
pursue  greater  degrees  of  certainty  and  predictability.  This  drove  the  cost  up  even  further, 


178 


and  injected  delays  into  the  project  as  project  leaders  spent  more  time  and  money  to 
ensure  a  successful  outcome. 

The  F-15’s  price  tag  also  limited  the  size  of  the  F-15  fleet,  resulting  in  relatively 
small  numbers  of  the  fighters  being  fielded.  As  the  ACEVAL/AIMVAL  wargame 
showed,  it  is  not  clear  the  smaller  fleet  of  F- 15s  would  actually  be  effective  against  a 
large  fleet  of  less  expensive,  less  capable  fighters  (i.e.  the  F-5).  After  the  disappointing 
results  of  these  1977  war  games  against  F-5’s,  such  tests  were  apparently  not  repeated. 

Despite  the  mixed  outcome  in  that  particular  assessment,  the  F-15  is  an 
outstanding  aircraft  with  a  proud  operational  history.  However,  it  initially  struggled  with 
poor  reliability,  and  performed  poorly  in  early  exercises  against  comparable  numbers  of  a 
simpler,  cheaper  fighter.  As  Tobias  et  al  pointed  out,  “if  there  is  anything  less  reliable 
than  a  complex,  sophisticated  weapon,  it  is  a  new  complex,  sophisticated  weapon.” 
(Tobias  et  al,  1982:369;  emphasis  in  original).  These  initial  shortcomings  were  eventually 
resolved  over  time,  at  considerable  expense. 

CO-EVALUATOR  ANALYSIS 

F=N/A 

1=0 

s=o 

T=0 

Outcome=A 
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PROJECT  NAME:  V-22  Osprey 
DATES:  1981  -2006 

STORY  SOURCE:  Thompson,  2007;  Cordesman  &  Kaeser,  2008;  Johnson,  1992;  Day, 
Stoetzer  and  Zamora,  2007;  Schinasi,  2001;  Olson,  2008. 

RUBRIC  SCORE:  -20 

F-Score:  -5 
I-Score:  -5 
S-Score:  -5 
T-Score:  -5 

OUTCOME:  A 
SAMPLE  STATEMENTS: 

FAST 

.  the  V-22  has  been  25  years  in  development,  more  than  twice  as  long  as  the  Apollo 
program  that  put  men  on  the  moon.”  (Thompson,  2007:1) 

“Originally,  the  program  was  designed  to  chum  out  the  first  of  more  thanl,000  tilt-rotors 
in  less  than  10  years  for  $40  million  each.”  (Thompson,  2007:2) 

“The  system  development  phase  took  25  years,  cost  over  $18  billion  and  claimed  the 
lives  of  30  people  in  four  crashes  of  flight  tests.”  (Cordesman  &  Kaeser,  2008:24) 

“In  2001,  a  Blue  Ribbon  panel. . .  found  the  aircraft  not  maintainable  or  ready  for 
operational  use.  It  recommended  continuing  the  program  at  a  minimal  rate  to  give  it  time 
to  sort  out  the  technical  problems.”  (Cordesman  &  Kaeser,  2008:25) 

“Renewed  testing  and  mechanical  adjustment  further  slowed  the  development  and 
increased  the  program’s  cost.”  (Cordesman  &  Kaeser,  2008:25) 

“Our  review  of  the  V-22  program,  which  is  already  in  low-rate  initial  production, 
revealed  that  the  Department  planned  to  proceed  with  a  full-rate  production  decision 
without  knowing  whether  new  technology  could  meet  Marine  Corps  requirements; 
whether  the  design  would  work  as  required;  or  whether  the  design  could  be  produced 
within  cost,  schedule  and  quality  targets...  Specifically,  developmental  testing  was 
deleted,  deferred,  or  simulated  in  order  to  meet  cost  and  schedule  goals.”  (Schinasi, 
2001:2) 

“Probes  into  the  deadly  2000  crashes  revealed  that  in  a  rush  to  deploy  the  aircraft,  the 
Marines  had  dangerously  cut  corners  in  their  testing  program.  The  number  of  different 
flight  configurations. . .  flown  by  test  pilots  to  ensure  safe  landings  was  reduced  by  half  to 
meet  deadlines.  Then  only  two-thirds  of  those  curtailed  flight  tests  were  conducted.” 
(Thompson,  2007:3) 

“We  weren’t  trying  to  rush  the  schedule.  We  were  event-driven,  not  schedule-driven.  We 
had  the  freedom  to  take  our  time.”  (Olson,  2008) 
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INEXPENSIVE 

when  procurement  requirement  estimates  exceeded  1,000  aircraft,  the  flyaway  cost 
per  aircraft  was  approximately  $2 1  million. . .  By  1986,  the  individual  flyaway  cost  was 
estimated  to  be  between  $28  million  and  $32  million. . .  Some  experts  suggest  that 
procurement  costs  could  reach  $50-$55  million  per  V-22  aircraft  by  the  time  the  aircraft 
reaches  production.”  (Johnson,  1992:18-19) 

“The  Pentagon  has  put  $20  billion  into  the  Osprey  and  expects  to  spend  an  additional 
$35  billion  before  the  program  is  finished.  In  exchange,  the  Marines,  Navy  and  Air  Force 
will  get  458  aircraft,  averaging  $119  million  per  copy.”  (Thompson,  2007:1) 

“The  aircraft  is  currently  five  times  more  expensive  than  the  system  it  replaces. . 
(Cordesman  &  Kaeser,  2008:24) 

“It  wasn’t  about  saving  a  bunch  of  money  -  it  was  about  doing  it.”  (Olson,  2008) 


SIMPLE 

“It  is  intended  to  execute  a  variety  of  missions  for  the  Marine  Corps,  the  Navy  and  the 
Air  Force,  including  troop  and  equipment  transport,  amphibious  assault,  search  and 
rescue,  and  special  operations.”  (Cordesman  &  Kaeser,  2008:24) 

“. . .  the  Navy  and  Marine  Corps  reduced  some  requirements  to  allow  the  aircraft  to  reach 
operational  readiness.”  (Cordesman  &  Kaeser,  2008:25) 

“We  had  to  do  so  many  fixes  so  late  in  the  design.”  (Olson,  2008) 

“It  was  very  organizationally  complex  initially,  which  led  to  communication  troubles.” 
(Olson,  2008) 


TINY 

“The  Osprey  is  also  three  times  heavier  than  the  helicopter  it  replaces.”  (Cordesman  & 
Kaeser,  2008:24) 

“Throughout  its  development,  however,  the  V-22  enjoyed  broad  and  persistent 
congressional  support.  The  V-22  program  has  nearly  2,000  suppliers  in  over  40  states  and 
created  jobs  in  276  congressional  districts.”  (Cordesman  &  Kaeser,  2008:26) 


OUTCOME 

“After  the  2000  grounding,  Osprey  pilots  were  told  to  fly  less  aggressively,  which  critics 
say  is  the  only  reason  no  V-22  has  crashed  since.  ‘They  keep  talking  about  all  the  things 
it  can  do,  but  little  by  little  its  operations  are  being  more  and  more  restricted,’  says  Philip 
Coyle,  who  monitored  the  V-22’s  development  as  the  Pentagon’s  top  weapons  tester 
from  1994  to  2001.”  (Thompson,  2007:3) 
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.  the  V-22  Osprey  arrives  in  Iraq  to  make  its  combat  debut — lacking  both  firepower 
and  the  ability  to  land  safely  if  it  loses  power  at  low  altitudes.”  (Cordesman  &  Kaeser, 
2008:24) 

“The  V-22  continues  to  suffer  problems  unusual  in  an  aircraft  that  first  flew  in  1989.  In 
March  2006,  for  example,  a  just-repaired  V-22  with  three  people  aboard  unexpectedly 
took  off  on  its  own  -  apparently  the  result  of  a  computer  glitch.”  (Thompson,  2007:6) 

“Ward  Carroll,  the  top  government  spokesman  for  the  V-22  program  from  2002  to  2005, 
believes  that  six  Ospreys,  about  5%  of  the  fleet,  will  crash  during  its  first  three  years  of 
operational  flight. . .  ‘I’m  still  not  convinced. . .  that  the  Marine  ground  pounders  are  in 
love  with  this  airplane.”  (Thompson,  2007:6) 

“...  rather  than  taking  advantage  of  the  V-22’s  long-range  self-deployment  capability, 
VMM-263  chose  to  ferry  them  over  on  the  USS  WASP  (LHD-1)  to  avoid  dangerous 
icing  conditions.  During  previous  trans-Atlantic  flights,  the  aircraft’s  de-icing  system  was 
shown  to  be  inadequate,  resulting  in  an  emergency  landing  in  Iceland  due  to  an  engine 
compressor  stall.”  (Day  et  al,  2007:  75) 

“Results  from  our  analysis  put  the  EH-101  on  top  in  both  the  net  present  value  and 
computer  simulation  models,  while  the  V-22  was  superior  in  measures  of  overall 
effectiveness. . .  Even  though  the  Osprey  does  offer  unique  capabilities  in  speed,  the 
benefit  over  traditional  helicopters  is  marginal.”  (Day  et  al,  2007:75) 

“We  got  unbelievable  reviews  by  Marines  and  Spec  Ops  users.  They  love  it. . .  I’ve  heard 
anecdotes  from  crew  chiefs,  fliers  and  other  riders  -  they  all  just  love  it.  They  refuse  to 
call  for  anything  else  once  they’ve  flown  in  it.”  (Olson,  2008) 

“On  its  third  Marine  deployment,  they  met  all  the  mission  metrics  and  flew  every  sortie 
they  were  scheduled  to  fly.”  (Olson,  2008) 


ANALYSIS 

The  V-22  Osprey  development  story  offers  no  evidence  the  project  leaders 
embraced  the  FIST  values.  Instead,  they  predominately  deemed  it  important  to  develop 
new,  cutting-edge  technologies,  regardless  of  the  time  or  cost  involved.  However,  as  with 
other  lengthy  development  projects,  it  is  worth  noting  that  the  values  expressed  by  project 
leaders  changed  over  the  years,  as  new  generations  of  leaders  took  the  helm  of  this 
project.  Although  the  changes  were  sometimes  positive,  this  values  flux  is  a  type  of 
instability  not  unlike  the  types  of  instability  discussed  elsewhere  in  this  paper.  Towards 
the  end  of  the  development,  project  leaders  seemed  to  lean  towards  ensuring  cost  and 
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schedule  goals  were  established  and  met,  but  this  seemed  to  be  for  political  reasons  rather 
than  because  being  Fast  and  Inexpensive  had  any  intrinsic  operational  or  technical  value. 

The  Osprey  development  showed  no  signs  of  the  Fast  value.  Not  only  did  the 
project  take  25  years  to  deliver  an  operational  capability,  but  in  the  few  instances  when 
project  leaders  attempted  to  accelerate  the  project,  they  did  so  by  cutting  comers  on  flight 
tests.  A  GAO  report  to  Congress  pointed  out  the  end  result  was  “knowledge  gained  in  V- 
22  program  falls  significantly  short  of  successful  programs.”  (Schinasi,  2001:20).  This 
willingness  to  cut  comers  and  “maintain  schedule”  without  actually  accomplishing  the 
necessary  tasks  is  evidence  of  speeding  and  is  not  an  indication  project  leaders  were 
genuinely  driven  by  the  Fast  value.  By  eliminating  essential  safety  tests,  the  project 
leaders  merely  created  the  impression  of  delivering  on  time.  In  truth,  the  system  they 
delivered  was  grossly  inadequate,  tragically  dangerous,  and  ultimately  required  expensive 
and  time-consuming  rework. 

There  is  also  no  evidence  project  leaders  embraced  the  Inexpensive  value.  The 
Osprey  program  began  in  1981  and  its  costs  were  already  escalating  significantly  by  1986. 
In  the  early  90’s,  more  than  a  decade  after  the  program  began,  estimates  of  the  Osprey’s 
per  aircraft  cost  were  conservatively  estimated  at  $50  million,  more  than  double  the 
initial  estimate.  By  the  time  it  became  operational  in  2006,  the  Osprey  actually  cost  $119 
million  per  copy  (Thompson,  2007),  which  supports  the  idea  that  project  leaders 
preferred  measures  of  merit  other  than  Inexpensive.  By2005,  project  leaders  seeking  to 
establish  a  new  direction  for  the  Osprey  promised  to  “deliver  on  time  and  high  quality 
and  at  a  certain  cost,”  but  only  after  securing  the  “freedom  to  take  our  time”  and  a 
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sizeable  budget.  (Olson,  2008).  This  approach  was  an  effective  and  appropriate  correction 
to  previous  “speeding”  behavior,  but  it  falls  short  of  being  evidence  of  the  Fast  value  at 
work. 

As  with  many  new  technologies,  the  Osprey  is  not  a  simple  machine,  and  there  is 
no  evidence  the  project  leaders  valued  Simplicity.  In  fact,  the  Osprey’s  complexity  and 
newness  is  widely  viewed  as  a  mark  of  sophistication  among  its  supporters  and  advocates. 
Simpler,  mature  alternatives  are  available,  such  as  the  EH- 101,  which  is  “already  flown 
by  military  organizations  around  the  world...  [and]  is  already  a  proven  commodity,” 
(Day  et  al,  2007:76).  Project  leaders  consistently  reject  these  alternatives  in  favor  of  the 
new,  exciting  “breakthrough”  tilt-rotor  capability.  This  preference  for  the  unknown  and 
unproven  over  reliable,  current  technology  shows  the  Simple  value  was  not  widely 
accepted  on  the  V-22  project. 

Overall,  V-22  project  leaders  showed  no  interest  in  the  Tiny  value.  The  Osprey  is 
large  and  heavy  when  compared  with  the  system  it  replaces.  Tellingly,  the  workforce  was 
spread  across  2,000  suppliers  in  40  states.  (Cordesman  &  Kaeser,  2008:26)  This  means 
the  Osprey  was  created  by  a  remarkably  complex  organization,  with  an  organizational 
footprint  that  involved  80%  of  the  states.  The  size  of  the  network  of  contributors 
increased  the  complexity  of  the  endeavor  as  well,  lowing  both  the  Osprey’s  S-score  and 
the  T-score. 

Operationally,  the  Osprey  has  clearly  under-delivered  on  several  fronts,  not  least 
of  which  is  its  rear-mounted  “peashooter”  gun.  Advocates  also  like  to  point  out  that  the 
V-22  “is  capable  of  self-deployment  to  virtually  any  area  on  earth”  (Hildreth,  1994:21), 
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although  the  validity  of  such  a  claim  made  in  1994  is  difficult  to  justify,  given  the  fact 
that  the  Osprey  did  not  become  operational  until  2006.  The  truth  is  that  the  self- 
deployment  capability  is  often  not  used  (Day  et  al,  2007),  despite  being  one  of  the 
Osprey’s  main  selling  points. 

Hildreth  claims  that  “During  Desert  Shield/Stonn  such  a  capability  [the  V-22] 
could  have  made  a  critical  difference  on  the  battlefield  and  on  a  hard-pressed  logistics 
system.”  (Hildreth,  1994:21).  Even  granting  the  conjecture  that  an  operational  Osprey 
would  have  helped  us  win  bigger,  the  fact  is  the  Osprey  was  15  years  too  late  to  be  of  use 
during  that  conflict.  No  doubt  a  large  fleet  of  working  V-22’s  would  have  been  extremely 
helpful  at  Normandy  on  D-Day,  but  it  does  no  good  to  say  a  system  would  have  been 
useful  if  it  had  been  ready.  The  fact  is,  it  was  not  ready  in  time  for  Desert  Storm,  and  so 
was  not  useful.  It  had  no  operational  impact  on  that  conflict. 

Given  the  length  of  time  the  Osprey  spent  in  development,  the  persistent  scaling- 
back  of  operational  requirements  and  the  restrictions  placed  on  its  use,  it  is  tempting  to 
judge  its  operational  outcome  a  failure.  However,  unlike  the  F-22,  the  V-22  has  actually 
flown  operational  missions  in  combat  zones  and  the  feedback  to  date  is  very  positive 
(Olson,  2008).  Thus,  the  V-22  is  placed  in  the  Success  column,  but  just  barely. 

It  must  be  noted  that  the  Osprey  currently  has  an  extremely  short  track  record  of 
operational  use.  The  fact  that  it  has  not  yet  established  much  of  a  record,  more  than  27 
years  after  the  project  was  begun,  is  a  significantly  negative  factor.  Over  the  coming 
years  the  Osprey  may  prove  itself  to  be  an  effective  component  of  American  military 
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strength  and  more  firmly  establish  itself  as  a  success,  or  its  limitations  may  come  to  light 
in  such  a  way  as  to  recommend  it  be  placed  in  the  Fail  column.  Time  will  tell. 
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PROJECT  NAME:  F-22  Raptor 
DATES:  1981  -2005 

STORY  SOURCES:  Muradian  and  Putrich,  2008;  Axe,  2006;  GlobalSecurity.org;  Hehs, 
1998;  Charette,  2008;  Walker,  2006;  Thompson,  2009;  Young,  2008. 

RUBRIC  SCORE:  -20 

F-Score:  -5 
I-Score:  -5 
S-Score:  -5 
T-Score:  -5 

OUTCOME:  F 
SAMPLE  STATEMENTS: 

FAST 

November  1981 :  Air  Force  identifies  need  for  advanced  tactical  fighter  to  replace  the  F- 
15 

September  1985 :  The  formal  ATF  request  for  proposal  is  issued. 

15  Dec  2005:  The  F-22  achieved  Initial  Operating  Capability.  (Global  Security). 

F-22  Acquisition  cycle  time  (months):  231  (Walker,  2006:59) 

“As  it  had  done  with  the  proposal  for  the  previous  phase,  the  Air  Force  delayed  the 
submittal  date  for  the  dem/val  [demonstration/validation]  proposals.  This  time  the  delay 
was  put  off  for  prototyping.”  (Hehs,  1998) 

“The  rollout  of  the  prototypes  was  initially  scheduled  for  mid  1989,  but  ongoing 
slippages  delayed  this.”  (Global  Security) 

“. . .  the  F-22  EMD  program  schedule  was  rephased  in  early  1993  and  again  in  mid  1994. 
The  F-22  program  was  rescheduled  due  to  funding  shortages.  The  rescheduling  resulted 
in  an  1 1 -month  delay  in  the  first  flight  (to  59  months  from  E&MD  start,  twice  as  long  as 
average  recent  experience)  and  an  18  month  delay  in  the  planned  Milestone  III  date. 
These  delays  should  not  be  construed  as  further  reducing  risks. . .”  (Global  Security) 

“The  F-22A  design  is  essentially  complete,  but  it  matured  slowly,  taking  over  3  years 
beyond  the  critical  design  review  to  meet  best  practice  standards.”  (Walker,  2006:60) 

“The  Air  Force  established  the  F-22A  modernization  and  improvement  program  in  2003” 
[two  years  before  IOC]  (Dodaro,  2008:59) 


INEXPENSIVE 

F-22  estimated  procurement  cost:  $32.14B  (FY06$)  for  181  aircraft 
Program  Unit  Cost:  $361.3M  per  aircraft  (Walker,  2006:59) 
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. .  in  late  1985,  the  USAF  sent  out  a  letter  to  the  competing  companies  to  encourage 
teaming. . .  The  program  was  going  to  be  expensive  and  big.”  (Hehs,  1998) 

“Technical  issues  are  not  the  biggest  challenge  for  the  F-22  program,  affordability  is.  Air 
Force  cost  estimates  indicated  the  program  could  exceed  the  Congressional  cap  of  $37.6 
billion  for  production  of  333  aircraft. . (Global  Security). 

“Since  1997  the  costs  of  avionics  has  increased  by  over  $951  million  or  24  percent  and 
problems  discovered  late  in  the  program  were  the  major  contributor.”  (Walker,  2006:60) 

“The  Air  Force  is  counting  on  $2.2  billion  in  future  cost  reduction  plans  to  offset 
estimated  cost  growth  and  enable  the  program  to  meet  the  latest  production  cost  estimate.” 
(Walker,  2006:60) 

“. . .  no  coherent  effort  has  been  made  to  contain  costs,  or  to  procure  a  cheaper  and  less 
sophisticated  version  of  the  aircraft.”  (Cordesman  &  Kaeser,  2008: 15) 


SIMPLE 

“In  addition  to  being  America’s  most  prominent  air-superiority  fighter,  the  F-22  evolved 
from  its  original  concept  to  become  a  lethal,  survivable  and  flexible  multi-mission 
fighter.  By  taking  advantage  of  emerging  technologies  the  F-22  has  emerged  as  a 
superior  platform  for  many  diverse  missions  including  intelligence  gathering, 
surveillance,  reconnaissance  and  electronic  attack.”  Lockheed  Martin  Website 
(http://www.lockheedmartin.com/products/f22/  accessed  10  Sep  08) 

“The  F-22  incorporated  revolutionary  advances  in  airframe,  low-observable  technology, 
maneuverability,  engines,  materials,  and  integrated  avionics  systems.”  (Global  Security). 

“The  three  critical  F-22A  technologies  (supercruise,  stealth,  and  integrated  avionics) 
appear  to  be  mature.  However,  two  of  these  technologies,  the  integrated  avionics  and 
stealth,  did  not  mature  until  several  years  after  the  start  of  development.”  (Walker, 
2006:60) 

“There  were  several  design  changes  required  to  the  aircraft  as  a  result  of  operational 
testing.”  (Walker,  2006:60) 

“Integrating  stealth,  speed,  and  maneuverability  became  the  fundamental  challenge  of  the 
ATF  program.  No  one  had  ever  attempted  such  a  complex  combination  before.”  (Hehs, 
1998) 

TINY 

“Maneuverability  requirements  tend  to  increase  the  size  of  the  wings  and  tails  and  make 
the  engines  bigger  than  necessary  for  supercruise  alone,  all  of  which  makes  stealth  more 
difficult  to  achieve.”  (Hehs,  1998) 
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OUTCOME 


“Defense  Secretary  Robert  Gates. . .  [pointed]  out  that  ‘the  F-22  has  not  perfonned  a 
single  mission’  in  either  Iraq  or  Afghanistan.”  (Charette,  2008:37) 

“Officials  believed  deploying  the  Raptor  so  near  Iran  would  cause  a  ‘strategic 
dislocation’  in  the  region,  sources  said.”  (Muradian  and  Putrich,  2008) 

“. .  .to  top  Pentagon  leaders,  sending  the  plane  into  the  Iraq  fight  was  overkill.  Loren 
Thompson  of  the  Lexington  Institute  agreed,  saying  the  F-22  is  ill-suited  to 
counterinsurgency  operations  in  Iraq.  ‘Most  of  what  the  F-22  brings  isn’t  useful  to 
fighting  insurgents,’  Thompson  said.”  (Muradian  and  Putrich,  2008) 

“. . .  the  Marine  Corps  aviators  of  All-Weather  Fighter  Attack  Squadron  332,  deployed  to 
A1  Asad  in  western  Iraq,  had  told  me  their  old  $40-million  F/A-18D  Hornet  equipped 
with  sensor  pods  are  better  suited  to  counter-insurgency  combat  than  the  $  130-million  F- 
22A  Raptors,  which  don’t  even  have  hard  points  for  pods.”  (Axe,  2006) 

“. . .  the  Air  Force  has  not  demonstrated  the  F-22A  can  achieve  its  reliability  goal  of  3 
hours  mean  time  between  maintenance.  It  does  not  expect  to  achieve  this  goal  until  the 
end  of  2009  when  most  of  the  aircraft  will  have  already  been  bought.”  (Walker,  2006:60) 

“The  fighter’s  purpose  is  described  largely  in  technical  tenns  rather  than  tied  to  a 
comprehensive  strategy  for  the  Air  Force. . .”  (Cordesman  &  Kaeser,  2008: 13) 

“[Secretary  of  Defense  Gates]  bluntly  stated  in  February  that  the  F-22  has  no  role  in  the 
war  on  terror.”  (Hoffman,  2008:1) 

“The  F-22s  that  exist  are  ready  to  fly  only  62%  of  the  time  and  haven’t  met  most  of  their 
performance  goals.”  (Thompson,  2009:25) 

“The  recent  mission  capable  data  for  FY2008  on  F-22s  had  a  mission  capable  rate 
somewhere  in  the  62  percent  range.  I  think  that's  troubling.  Follow-on  operation  tests  in 
2007  raised  operational  suitability  issues  and  noted  that  the  airplane  still  does  not  meet 
most  of  its  KPPs.  It  meets  some,  but  not  all.  Key  perfonnance  parameters.  The  trend  in 
those  operational  tests,  there  was  an  IOT&E,  a  follow-on  test  I  think  in  2004  and  a 
follow-on  test  in  2007.  The  trend  is  actually  negative.  The  maintenance  man  hours  per 
flying  hour  have  increased  through  those  tests.  The  last  one  was  a  substantial  increase. 
The  airplane  is  proving  very  expensive  to  operate,  not  seeing  the  mission  capable  rates 
we  expected.”  (Young,  2008) 


ANALYSIS 

Like  the  Comanche  development  team,  F-22  project  leaders  clearly  were 
committed  to  developing  innovative  new  technology  rather  than  delivering  a  simple, 
effective  system  on  schedule.  Despite  persistent  perfonnance  shortfalls,  the  team’s 
primary  value  seemed  to  be  incorporation  of  complex,  high-tech  solutions.  The  project 
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leaders  apparently  did  not  deem  it  important  to  deliver  a  low-cost,  simple  solution  on  a 
short  timeline,  and  were  willing  to  sacrifice  cost,  schedule  and  usability  in  order  to 
integrate  newer,  unproven  technologies. 

In  terms  of  operational  outcome,  a  strong  case  can  be  made  that  the  Raptor  is  a 
failure  so  far.  The  F-22A  was  originally  envisioned  to  counter  “two  hypothetical  Soviet 
aircraft,”  (Charette,  2008:36).  However,  the  Soviet  Union  collapsed  in  1991,  fourteen 
years  before  the  F-22A  declared  IOC,  so  its  effectiveness  against  that  particular  threat 
was  zero.  It  can  be  argued  that  the  US  must  still  deal  with  the  weapons  produced  by  the 
former  USSR  (and  current  Russian  Federation)  as  they  are  propagated  throughout  the 
world,  but  its  effectiveness  against  those  threats  is  currently  unproven.  In  fact,  the  very 
presence  of  those  threats  is  also  questionable.  Similarly,  the  assertion  that  currently 
available  aircraft  (or  hypothetically  upgraded  versions  the  Air  Force  could  have 
developed  but  didn’t)  are  unable  to  meet  those  threats  has  also  not  been  proven. 

Along  with  its  failure  to  demonstrate  it  meets  reliability  goals,  the  F-22A  has  been 
deemed  ineffective  in  fighting  insurgents,  the  primary  threat  currently  facing  US  forces. 
It  has  also  been  judged  politically  unsuitable  for  deployment  near  Iran,  for  fear  of 
unnecessarily  provoking  that  country.  Although  IOC  was  declared  in  2005,  the  F-22  has 
not  flown  a  single  mission  over  Afghanistan  or  Iraq  as  of  March  2009.  These  are 
significant  operational  shortfalls,  and  are  the  true  measure  of  a  program’s  success. 

Physically,  the  F-22  A  apparently  outperforms  all  current  fighter  aircraft  in  the 
world  today,  but  this  project  assesses  Operational  success,  not  technical  success.  Weapon 
systems  are  supposed  to  bring  capabilities  to  the  field,  not  simply  supersede  existing 
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systems  already  in  the  inventory.  In  fact,  considering  the  fact  that  F-15’s  routinely  fly 
actual  combat  missions  and  F-22’s  do  not,  one  might  ask  in  what  sense  the  F-22  is  the 
superior  aircraft.  Since  the  Raptor  has  yet  to  fly  any  operational  missions  four  years  after 
going  IOC,  despite  US  involvement  in  two  concurrent  wars,  the  only  possible 
“operational  impact”  grade  it  can  receive  is  an  F  for  failure. 

Further,  as  the  Air  Force  Studies  Board  explained,  “Programs  that  failed,  in  the 
committees  view,  may  have  delivered  successful  products  but  were  well  outside  the 
reasonable  expectations  of  the  original  program  and  were  only  successful  in  delivering 
products  after  the  addition  of  substantial  unplanned  funding  and  a  substantial  extension  of 
the  original  schedule.”  (AF  Studies  Board,  2008:26)  This  clearly  describes  the  situation 
for  the  F-22. 

Unlike  the  Comanche  helicopter  or  A- 12  Avenger,  the  F-22  team  actually 
delivered  operational  aircraft  and  achieved  IOC.  This  opens  the  possibility  of  it 
eventually  being  used  in  combat,  should  the  right  kind  of  war  come  along,  and  so  it  could 
become  something  other  than  an  operational  failure.  Despite  its  current  low  availability 
rate  (62%),  there  is  every  indication  it  would  perfonn  admirably  at  such  a  time  as  its 
capabilities  were  needed.  If  so,  the  F-22’s  grade  could  be  changed  from  an  F  to  an  A. 
However,  whether  or  when  its  capabilities  will  be  needed  or  effective  in  future  combat 
situations  remains  to  be  seen.  The  fact  that  it  has  not  yet  established  a  positive  operational 
track  record  in  the  years  since  it  went  IOC  requires  me  to  give  it  an  F. 
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PROJECT  NAME:  Comanche  Helicopter 
DATES:  1983  -2004 

STORY  SOURCE:  Wayne,  2003;  Merle,  2004;  Congressional  Budget  Office,  1997; 
Bender,  2004;  Global  Security,  2008. 

RUBRIC  SCORE:  -20 

F-Score:  -5 
I-Score:  -5 
S-Score:  -5 
T-Score:  -5 

OUTCOME:  F.  Cancelled  in  2004,  delivered  0  aircraft 

SAMPLE  STATEMENTS: 

FAST 

“It  had  a  long  and  troubled  history. . .”  (Merle,  2004:A1) 

“The  helicopter  is  still  in  the  development  stage,  which  will  continue  at  least  through 
2004.  As  recently  as  1992,  the  Army  had  planned  to  start  buying  Comanches  in  1996,  but 
it  has  since  delayed  the  start  of  production  until  2005.”  (CBO,  1997) 

“Skeptics  of  the  program  suggested  that  unmanned  planes  capable  of  perfonning  the 
Comanche’s  surveillance  and  precision-strike  role  will  be  available  to  the  Army  prior  to 
the  maturing  of  the  Comanche  system.”  (Global  Security,  2008) 

“The  first  helicopter  will  be  combat-ready  in  September  2009,  three  years  behind  the 
previous  schedule.  Production  and  purchase  of  the  first  helicopters  would  begin  in  fiscal 
2006,  one  year  later  than  planned  under  the  previous  schedule.”  (Global  Security,  2008) 


INEXPENSIVE 

“The  utility  version  was  dropped  in  1988,  however,  because  the  program  had  become  too 
costly.”  (CBO,  1997) 

“The  Anny  would  have  spent  $14  billion  on  the  commence  program  through  2011 
without  getting  aircraft  significantly  more  capable  than  the  upgraded  Apaches  it  already 
plans  to  buy,  Anny  officials  said.”  (Merle,  2004:  A2) 

“These  changes  in  the  objective  and  size  of  the  program  have  caused  the  cost  of  each 
Comanche  helicopter  -  expressed  in  1997  dollars  -  to  more  than  double  since  the 
program  began,  from  $11  million  in  1985  to  $26  million  based  on  the  Army’s  1996 
estimate.  Furthennore,  the  Comanche  has  become  more  expensive  to  acquire  than  the 
Anny’s  current  generation  of  attack  helicopter.”  (CBO,  1997). 

“The  new  Acquisition  Decision  Memorandum  (ADM)  fonnally  approving  the  plan  added 
about  $3.4  billion  to  the  Comanche’s  $3.1  billion  development  program.”  (Global 
Security,  2008). 
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“Matt  Broder,  a  spokesman  for  Sikorsky,  in  response  to  a  request  for  comment,  said, 
"Our  company's  policy  is  not  to  respond  to  rumors.  As  far  as  we  are  concerned,  the 
program  is  fully  funded  and  on  schedule."  Boeing  officials  said  the  Chicago-based 
company  had  not  been  informed  of  any  program  cancellation.  "The  Comanche  program  is 
on  track  and  schedule.  We  have  not  been  notified  about  any  change  in  the  program 
status,"  said  Boeing  spokesman  John  Morrocco.”  (Dunham  &  Emery,  2004) 

SIMPLE 

“Government  watchdog  groups  lauded  the  [cancellation]  decision,  saying  the  program 
has  long  been  plagued  with  cost  overruns,  schedule  delays,  and  an  overzealous  desire  to 
include  unproven  technologies.”  (Bender,  2004) 

“Some  analysts  argue  that  the  threats  the  Comanche  is  likely  to  face  would  not  demand 
the  very  sophisticated  stealth,  avionics,  and  aeronautic  technologies  slated  for  the  new 
helicopter.”  (CBO,  1997). 

“According  to  the  DOT&E,  technical  challenges  remained  for  software  integration  and 
testing  of  mission  equipment,  weight  reduction,  radar  signatures,  antenna  performance, 
gun  system  performance,  and  aided  target  detection  algorithm  performance.”  (Global 
Security,  2008). 


TINY 

“Empty  weight  projections  for  Block  I,  II  and  III  aircraft  were  slightly  higher  than  weight 
goals  for  each  block.”  (Global  Security,  2008) 


OUTCOME 

“The  Army  yesterday  cancelled  development  of  the  $39  billion  Comanche  helicopter 
after  2 1  years  of  escalating  costs,  technical  glitches  and  redesigns  during  a  program  that 
failed  to  produce  a  single  operational  aircraft.”  (Merle,  2004: A 1) 

“. . .  the  Comanche,  which  was  conceived  at  the  height  of  the  Cold  War,  will  no  longer 
need  to  counter  threats  of  the  same  scale  or  sophistication  as  those  it  was  designed  to 
thwart.”  (CBO,  1997). 

“Some  analysts  argue  that  the  threats  the  Comanche  is  likely  to  face  would  not  demand 
the  very  sophisticated  stealth,  avionics,  and  aeronautic  technologies  slated  for  the  new 
helicopter.”  (CBO,  1997). 

“Based  on  analysis  by  DOT&E  of  current  designs  and  limited  testing  on  a  contractor 
range,  the  radar  warning  receiver  and  two  communications  antennas  did  not  meet  some 
RCS  allocations.  The  satellite  receiver  antenna  did  not  meet  performance  requirements, 
and  the  radar  warning  sensitivity  was  marginal.  Efforts  to  reduce  the  RCS  would  have 
been  likely  to  adversely  affect  the  performance  of  already  marginal  antennas.”  (Global 
Security,  2008). 
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ANALYSIS 

The  Comanche  project  was  clearly  committed  to  developing  innovative  new 
technology.  Despite  persistent  performance  shortfalls,  the  project  leaders’  primary  value 
seemed  to  be  incorporation  of  complex,  high-tech  solutions.  There  is  no  evidence  project 
leaders  deemed  it  important  to  deliver  a  low-cost,  simple  solution  on  a  short  timeline. 
Rather,  the  evidence  suggests  they  were  more  than  willing  to  sacrifice  cost,  schedule  and 
usability  in  order  to  ensure  the  Comanche’s  technologies  were  cutting-edge,  however 
unproven  or  unreliable. 

While  some  optimistic  commentators  initially  referred  to  it  as  “the  centerpiece  of 
America's  21st  century  war-fighting  strategy”  (Moore,  1998)  the  Comanche  was  clearly  a 
failure  on  every  level.  Although  officials  from  Boeing  and  Sikorsky  insisted  the  program 
was  “on  track  and  on  schedule”  up  until  the  day  it  was  cancelled,  the  actual  cost  and 
schedule  far  exceeded  estimates.  More  importantly,  the  aircraft  did  not  perfonn  as 
required.  Ultimately,  the  external  threat  changed  to  such  a  degree  that,  even  if  it  had 
worked,  it  would  have  proven  operationally  unnecessary.  The  project  was  cancelled  after 
spending  nearly  $7B  over  two  decades.  The  cancellation  enabled  the  Pentagon  to  avoid 
spending  the  additional  $14B  budgeted  to  continue  developing  and  purchasing  the 
aircraft. 
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PROJECT  NAME:  Future  Imagery  Architecture  (FIA) 

DATES:  1996  -  Sept  2005 

STORY  SOURCE:  Taubman,  2007;  Anselmo  &  Butler,  2005;  Pae,  2001 
RUBRIC  SCORE:  -20 

F-Score:  -5 
I-Score:  -5 
S-Score:  -5 
T-Score:  -5 

OUTCOME:  F  -  Project  Cancelled  Sept  2005 

SAMPLE  STATEMENTS: 

FAST 

“The  panel  reported  that  the  project,  called  Future  Imagery  Architecture,  was  far  behind 
schedule  and  would  most  likely  cost  $2  billion  to  $3  billion  more  than  planned.” 
(Taubman,  2007:  1) 

. .  the  contract  would  discourage  overruns  or  delays  with  financial  penalties.”  (Taubman, 
2007:7) 

“The  National  Reconnaissance  Office  announced  its  decision  on  Sept.  3,  1999,  after 
studying  the  bids  for  nearly  a  year.”  (Taubman,  2007:8) 

“As  costs  escalated,  Boeing  cut  back  on  testing...  If  a  component  failed,  Boeing,  lacking 
a  backup  approach,  had  to  return  to  square  one,  forcing  new  delays.”  (Taubman,  2007:9) 

“The  electro-optical  satellites...  are  close  to  five  years  behind  schedule.”  (Anselmo  & 
Butler,  2005:1) 


INEXPENSIVE 

“...  the  program  was  threatening  to  outstrip  its  $5  billion  budget.”  (Taubman,  2007:1) 

“...  they  advised  Mr.  Teats  to  seek  an  infusion  of  $700  million.”  (Taubman,  2007:1) 

“By  the  time  the  project,  known  by  its  initials  F.I. A.,  was  killed  in  September  2005  -  a 
year  after  the  first  satellite  was  originally  to  have  been  delivered  -  cost  estimates  ran  as 
high  as  $18  billion.”  (Taubman,  2007:2) 

“...  Team  377  [Boeing’s  secret  planning  group]  requested  $100  million  just  to  draft  the 
proposal...”  (Taubman,  2007:6) 

“F.I.  A.  was  grossly  underfunded.”  [said  Mr.  Young,  chainnan  of  one  of  the  panels  that 
examined  the  project]  (Taubman,  2007:7) 

“. . .  the  satellite,  under  pressure  from  Congress  to  control  costs,  would  no  longer  have  a 
reserve  fund.”  (Taubman,  2007:7) 
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“I  took  what  it  had  cost  to  build  a  comparably  complex  system  before,  figured  in  inflation, 
and  realized  the  project  would  cost  $4  billion  more  than  the  government  had  planned  and 
Boeing  was  proposing.”  (Taubman,  2007:7) 

“The  cost  of  an  overrun  was  so  ruinous  that  the  strongest  incentive  it  provided  to  the 
contractor  was  to  prove  they  were  on  cost.”  (Taubman,  2007: 10) 

“...  the  government  had  approved  an  additional  $3.6  billion.  Still,  rather  than 
recommending  cancellation,  the  Young  panel  said  the  program  could  be  salvaged  with 
even  more  financing  and  changes  in  the  program  and  schedule.”  (Taubman,  2007:10) 

“...  the  system’s  price  tag  has  grown  from  $6  billion  to  at  least  $15  billion.”  (Ansehno  & 
Butler,  2005:1) 


SIMPLE 

“But  perhaps  more  striking  is  that  the  multiple  failures  that  led  to  the  program’s  demise 
reveal  weaknesses  in  the  govermnent’s  ability  to  manage  complex  contracts...” 
(Taubman,  2007:1) 

“. . .  an  internal  assessment  that  questioned  whether  its  lofty  technological  goals  were 
attainable  given  the  tight  budget  and  schedule.”  (Taubman,  2007:2) 

“...  the  satellite  agency,  hobbled  by...  the  loss  of  seasoned  staff  members,  lacked  the 
expertise  to  make  sound  engineering  evaluations  of  its  own...”  (Taubman,  2007:2) 

“The  satellites  were  loaded  with  intelligence  collection  requirements,  as  numerous 
intelligence  and  military  services  competed  to  influence  the  design.  Boeing’s  initial 
design  for  the  optical  system  that  was  the  heart  of  one  of  the  two  new  satellite  systems 
was  so  elaborate  that  optical  engineers  working  on  the  project  said  it  could  not  be  built.” 
(Taubman,  2007:2) 

“A  torrent  of  defective  parts...  repeatedly  stalled  work.”  (Taubman,  2007:2) 

“The  leaders  of  Team  377  realized  that  the  best  hope  of  impressing  the  satellite  agency 
was  to  design  a  system  that  was  cheaper  and  better — more  technologically  daring — than 
anything  Lockheed  might  propose.”  (Taubman,  2007:6) 

“The  goal. . .  was  a  revolutionary  zoom  lens. . .  As  for  the  radar- imaging  satellite,  Boeing 
designed  a  relatively  simple  system  with  one  major  exception: ...  it  would  produce  a  far 
stronger  radar  signal  than  any  previous  satellite  had.  Pulling  off  such  complex  new 
technology  typically  requires  extensive  testing  and  work  on  multiple  solutions  to 
especially  difficult  problems.”  (Taubman,  2007:6-7) 

“. . .  outside  engineers  questioned  the  photo  satellite’s  intricate  optical  system. . .  it  soon 
became  clear  the  system  could  not  be  built.”  (Taubman,  2007:8-9) 

“. . .  the  government’s  addition  of  more  stringent  requirements,  the  new  satellites  are 
looking  more  like  the  ‘Battlestar  Galacticas’  they  were  supposed  to  replace  than  the 
simple  spacecraft  envisioned  in  1999.”  (Ansehno  &  Butler,  2005: 1) 
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TINY 

“The  endeavor  will  require  5,000  engineers,  technicians  and  computer  programmers  over 
the  next  five  years,  and  that  will  just  be  for  the  initial  design  and  development.”  (Pae, 
2001:1) 

OUTCOME 

“It  took  two  more  years,  several  more  review  panels,  and  billions  more  dollars  before  the 
government  finally  killed  the  project  -  perhaps  the  most  spectacular  and  expensive  failure 
in  the  50-year  history  of  American  spy  satellite  projects.”  (Taubman,  2007:1) 


ANALYSIS 

Anselmo  &  Butler  claim  that  “FIA...  was  procured  under  the  faster-better- 
cheaper  mindset  of  the  1990’s,”  (Anselmo  &  Butler,  2005:2),  and  early  reports  describe 
the  project  in  terms  of  using  smaller,  cheaper  satellites.  For  example,  Taubman  writes: 


“In  1996,  a  commission  created  by  the  director  of  central  intelligence 
recommended  building  a  fleet  of  light,  small,  relatively  inexpensive  satellites... 
The  panel  also  envisioned  saving  money  and  time  by  taking  advantage  of 
technologies  and  parts  developed  by  commercial  satellite  companies...  Congress 
demanded  rigid  spending  guidelines  for  the  satellite  project...”  (Taubman,  2007:4) 

Despite  this  initial  vision,  it  is  not  clear  the  project  leaders  really  accepted  or 
understood  these  values.  In  fact,  despite  congressional  pressure  and  contractual  clauses 
designed  to  minimize  cost  and  schedule,  neither  the  government  personnel  nor  the 
contractors  seemed  to  have  a  favorable  opinion  of  the  FIST  value  set. 


Looking  back,  many  of  the  involved  parties  blame  FIA’s  failure  on  the  strict  cost 
and  schedule  constraints,  as  if  to  say  they  could  have  succeeded  if  they  had  only  been 
allowed  to  overrun  the  budget  and  schedule.  This  is  the  antithesis  of  the  FIST  values. 
Despite  the  frequent  claims  that  allowing  overruns  would  have  solved  FIA’s  problems, 
the  project  did  quickly  overrun  both  cost  and  schedule,  to  no  avail,  making  that  assertion 
seem  spurious.  But  whether  it  is  a  valid  complaint  or  not,  the  project  leaders’  willingness 
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to  attribute  FIA’s  failure  to  inadequate  funding  and  insufficient  time  is  clear  evidence  that 
the  team  rejected  the  Fast  and  Inexpensive  values.  Project  leaders  who  embrace  the  FIST 
values  think  it  is  good  and  important  to  have  a  short  schedule  and  a  small  budget  and 
view  these  constraints  as  elements  of  success.  Project  leaders  who  value  FIST  never  say 
“We  would  have  succeeded  if  we’d  had  a  lot  more  time  and  money.” 

It  is  worth  pointing  out  that  FIA’s  attempt  at  faster-better-cheaper  (FBC) 
technology  came  in  the  late  1990’s,  when  NASA  was  experiencing  similar  problems  with 
their  own  FBC  initiatives.  Both  organizations’  problems  can  be  traced  to  two  primary 
sources:  complexity  and  fonnalization.  In  their  late  1990’s  missions,  NASA  and  the  NRO 
seem  to  have  under-appreciated  the  role  of  simplicity  in  FBC  projects,  as  applied  to  the 
technology  under  development  and  the  organizations  themselves.  Both  NASA  and  the 
NRO  attempted  to  build  highly  complex  systems  with  short  schedules  and  small  budgets, 
and  attempted  to  turn  the  FBC  approach  into  a  mechanical,  process-oriented, 
contractually-driven  checklist  rather  than  a  humanistic,  talent-oriented,  value-driven 
effort.  This  is  the  opposite  of  the  Simple  value,  and  is  not  consistent  with  the  successful 
implementations  of  the  FBC  initiative. 

Writing  about  NASA’s  failures  with  FBC  missions  in  1999,  McCurdy  explains 
“proponents  of  the  approach  created  failure  when  they  reduced  cost  and  schedule  faster 
than  they  lessened  complexity.”  (McCurdy,  2001:25)  The  NRO  appears  to  have  made  a 
similar  error  with  FIA.  Project  leaders  expected  to  reduce  cost  and  schedule  without 
reducing  complexity.  They  then  blamed  their  failures  on  inadequate  resources  and  began 
to  overrun.  Had  they  really  understood  and  implemented  the  FIST  values,  they  would 
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have  looked  for  opportunities  to  simplify,  streamline  and  shrink  the  system,  seeking  to 
improve  outcomes  by  reducing  complexity  rather  than  seeking  additional  money  and  time. 

Further,  the  FBC  and  FIST  approaches  tends  to  have  a  higher  failure-per-attempt 
rate  than  traditional  approaches,  so  they  often  requires  multiple  attempts.  These  multiple 
attempts  are  possible  because  of  the  time  and  money  savings  associated  with  a  FIST 
project.  FIA  expected  a  complex  organizational  structure  to  deliver  a  complex  system 
with  a  low  cost  and  short  schedule,  and  get  it  right  the  first  time.  The  idea  of  experiencing 
a  failure  and  having  to  start  over  was  unacceptable.  This  is  not  consistent  with  the  FIST 
values. 

Along  with  ignoring  the  Simple  value  and  rejecting  the  Inexpensive  value,  Boeing 
seriously  misunderstood  the  Fast  value.  This  led  them  to  fall  into  a  classic  speeding  trap, 
in  which  they  cut  back  on  early  testing  in  the  interest  of  faster  delivery.  When  the  system 
failed  tests  later  in  development,  the  delays  were  extensive  and  expensive,  far  exceeding 
the  cost  or  time  associated  with  doing  the  early  tests  in  the  first  place.  This  short-sighted 
perspective  on  the  value  of  speed  has  been  lampooned  since  Aesop’s  story  of  the  tortoise 
and  the  hare,  and  Boeing  should  clearly  have  known  better. 

Another  significant  aspect  of  the  problems  was  excessive  optimism  on  the  part  of 
Boeing  officials.  The  term  “excessive  optimism”  is  actually  a  euphemism  for  dishonesty, 
and  the  failure  to  be  honest  about  the  project’s  status  contributed  significantly  to  the 
eventual  outcome.  Taubman  writes: 

“Boeing’s  point  man  on  the  job  was  Ed  Nowinsky...  he  acknowledged  that 

Boeing  frequently  provided  the  government  with  positive  reports  on  the  troubled 
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project...  Mr.  Nowinski  said,  ‘but  it  was  certainly  in  my  best  interest  to  be  very 
optimistic  about  what  we  could  do.’”  (Taubman,  2007:2) 

This  is  further  evidence  that  the  FBC  approach  was  mere  window  dressing,  and 
not  evidence  of  the  project  leader’s  actual  values.  Boeing  and  NRO  leaders  were 
detennined  to  maintain  the  appearance  of  being  on  time  and  on  budget,  all  the  while 
increasing  complexity  and  seeking  opportunities  to  add  additional  funding  and  time  to  the 
FIA  project. 

In  the  end,  FIA  was  cancelled,  earning  a  description  as  “the  most  spectacular  and 
expensive  failure  in  the  50-year  history  of  American  spy  satellite  projects.”  (Taubman, 
2007:1) 
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PROJECT  NAME:  Division  Air  Defense  System  (DIVAD)  aka  Sergeant  York 
DATES:  1977  -  1985 

STORY  SOURCE:  Ditton,  1988; 

RUBRIC  SCORE:  -20 
F-Score:  -5 
I-Score:  -5 
S-Score:  -5 
T-Score:  -5 
OUTCOME:  F 
SAMPLE  STATEMENTS: 

FAST 

“Instead  of  the  normal  ten  to  fifteen  years  needed  to  bring  a  defense  weapon  system 
through  research  and  development  to  the  production  phase,  seven  years  was  allotted  for 
the  DIVAD  program.”  (Ditton,  1988:3). 

“There  was,  however,  no  flexibility  in  the  June  1980  date  when  the  first  prototypes  were 
to  be  delivered  to  Fort  Bliss,  Texas.”  (Ditton,  1988:3) 

“Both  contractors,  after  more  than  two  years  of  work,  unveiled  their  prototypes  on 
schedule...  But  problems  were  already  evident.  ‘Prototypes  from  both  contractors  were 
unexpectedly  immature,’  according  to  a  1986  General  Accounting  Office  report.”  (Adam, 
1987:30) 

“. . .  the  fixed-price  options. . .  put  pressure  on  decision  makers  to  proceed  with  production 
on  schedule,  despite  technical  difficulties.”  (Ditton,  1988:7) 

“. . .  greater  priority  has  been  given  to  adhering  to  the  schedule  than  to  correcting  some 
serious  system  performance  problems  at  this  time.”  (Ditton,  1988:8) 

“The  ambitious  goal  was  to...  achieve  operational  capability  by  October  1983 — less  than 
six  and  a  half  years  from  the  start.  This  was  fast  compared  with  most  other  Anny 
programs.  It  took  about  eight  years  to  develop  the  M-l  tank,  for  example.”  (Adam, 
1987:30) 

“Colonel  Russell  W.  Parker,  project  manager  of  Army  air  defense  gun  systems,  stated. . . 
‘We  expect  this  somewhat  unorthodox  approach  to  permit  a  much  reduced  development 
time,  thus  resulting  in  an  earliest  fielding  date,  albeit  with  higher  but  accepted  risks.’  The 
additional  risks  stemmed  from  reducing  the  required  testing  and  combining  the  gun’s 
follow-on  development  with  initial  production.”  (Adam,  1987:30) 

“. .  .calendar  schedule  compliance  had  a  higher  priority  than  test  matrix  completion. . .  the 
combination  of  these  factors  precluded  complete  execution  of  the  test  plan.”  (Adam, 
1987:33) 
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INEXPENSIVE 


. .  a  fixed-price  incentive  type  contract. . .  incorporated  twelve  firm  requirements  and 
forty-three  other  requirements,  which  the  contractor  could  trade  off  for  cost  and  schedule 
benefits.”  (Ditton,  1988:4) 

“The  Anny  successfully  controlled  costs  until  contract  termination.”  (Ditton,  1988:7) 

“Such  high-tech  guns,  costing  more  than  $6  million  apiece,  or  more  than  twice  the  cost  of 
anM-1  tank...”  (Adam,  1987:29) 

“[BGen]  Cannon  [chairman  of  the  DIVAD  source  selection  board]  reminded  the  General 
Dynamics  team  that  the  paramount  evaluation  factors  were  system  performance  and  cost.” 
(Adam,  1987:30) 


SIMPLE 

“...  the  Anny  adopted  a  ‘hands  off  policy.  The  two  contractors  competing  for  the 
production  contract  were  free  to  develop  a  DIVAD  in  any  manner  they  chose,  provided 
that  the  specifications  were  met.”  (Ditton,  1988:4) 

“. . .  companies  were  given  maximum  flexibility  and  were  guaranteed  minimum  meddling 
by  the  Government.”  (Adam,  1987:30) 

“. . .  an  amalgam  of  existing  components  put  together  under  an  accelerated  program. . .” 
(Adam,  1987:28) 

“Another  twist  to  the  acquisition  strategy  was  the  use  of  off-the-shelf  components  to 
build  the  gun.”  (Adam,  1987:30) 

“. . .  the  DIVAD  gun  system  was  supposed  to  be  an  integration  of  proven  major 
components,  including  the  M-48A5  tank  chassis,  twin  Swedish  Bofors  40  millimeter 
guns,  and  radars  from  the  F-16  fighter.”  (Ditton,  1988:4) 

“. . .  the  specifications  were  unworkable  because  they  attempted  the  technologically 
impossible.  Although  each  major  subsystem  was  a  proven  component,  the  sum  of  the 
components  could  not  match  contract  requirements,  much  less  battlefield  reality.  The  F- 
16  radar  operates  on  detection  of  movement  and  was  successful  at  acquiring  moving 
targets.  Unfortunately,  it  had  difficulty  acquiring  stationary  targets.  The  computer  fire 
direction  system  could  not  adequately  track  moving  targets  because  it  could  not  anticipate 
where  a  moving  aircraft  would  go  next.”  (Ditton,  1988:6) 

“. . .  most  of  the  software  for  the  York  had  to  be  newly  developed. . .  ‘Errors  were 
consistently  observed  at  all  levels  of  software  testing.’”  (Adam,  1987:32) 


TINY 

“Its  goal  was  to  develop  a  bigger  and  better  gun  than  the  Gepard  [a  German  air  defense 
gun]”  (Adam,  1987:29) 


202 


“Greater  ranges  would  require  larger  projectiles,  which  cannot  be  fired  as  quickly  as 
smaller  ones.  For  example  the  20-mm  Vulcan  fires  3000  rounds  a  minute;  the  40-mm 
York  fired  only  60.”  (Adam,  1987:29) 


OUTCOME 

“. . .  on  August  27,  1985,  the  Secretary  of  Defense  canceled  the  DIVAD  procurement. . . 
Sixty-five  DIVADs  had  already  been  built,  and  1.8  billion  dollars  had  been  spent  on  the 
program.”  (Ditton,  1988:6) 

“The  DIVAD  procurement  is  an  excellent  example  of  many  of  the  problems  with  modern 
weapon  acquisition  programs.”  (Ditton,  1988:6) 

“. . .  for  the  guns  to  shoot  down  nine  of  the  1 1  targets,  many  of  the  aircraft  had  to  make 
multiple  passes.”  (Adam,  1987:33) 


ANALYSIS 

The  Anny’s  DIVAD  program  has  all  the  appearances  of  a  FIST  project,  with 
frequent  assertions  in  support  of  several  of  the  FIST  values  and  many  instances  of 
contractual,  procedural  and  technical  decisions  being  made  based  on  a  desire  to  cut  cost 
and  schedule.  The  contractor  was  given  the  flexibility  to  trade-off  requirements  for  cost 
and  schedule  savings,  which  seems  to  indicate  that  Fast  and  Inexpensive  mattered  more 
than  high-technology  or  performance.  However,  upon  closer  examination,  this  project  is 
revealed  to  be  a  pseudo-FIST  effort,  with  a  significant  emphasis  on  appearing  to  be  fast 
and  inexpensive,  rather  than  actually  embracing  or  understanding  how  to  apply  these 
values. 

At  first  glance,  the  fact  that  the  DIVAD  team  was  given  an  aggressive  timeline 
seems  to  indicate  the  presence  of  the  Fast  value.  However,  the  project  leaders’ 
determination  to  avoid  a  schedule  slip  led  them  to  ignore  “serious  system  performance 
problems.”  While  the  team  achieved  a  high  degree  of  “calendar  schedule  compliance,” 
one  might  ask  what  it  means  to  be  on  schedule  if  the  system  does  not  actually  do  what  it 
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needs  to  do.  A  project  that  fails  to  meet  the  critical  perfonnance  requirements  by  the 
necessary  date  cannot  be  considered  “on  schedule.”  That  is,  the  program  might  have  been 
on  schedule,  simply  because  June  followed  May,  but  the  system  clearly  was  not.  And 
while  the  Fast  value  allows  project  leaders  to  trim  unnecessary  activities,  the  DIVAD 
project  skipped  the  part  where  they  were  supposed  to  build  something  that  works,  and 
ended  up  merely  speeding. 

Similarly,  the  literature  asserts  that  project  leaders  “successfully  controlled  costs” 
on  the  DIVAD  development.  This  is  not  an  accurate  statement.  Project  leaders  merely 
controlled  spending,  not  cost,  which  is  a  horse  of  a  different  color.  That  is,  they  spent  as 
much  as  they’d  planned  to  spend  in  the  time  allotted,  but  they  purchased  far  less  than 
their  customers  needed.  By  purchasing  less,  they  were  not  genuinely  expressing  the 
Inexpensive  value.  They  were  expressing  something  entirely  different,  an  ugly  twin  that 
appears  similar  from  a  distance  but  is  fundamentally  dissimilar. 

The  Earned  Value  Management  (EVM)  approach  demonstrates  the  difference 
between  controlling  spending  and  controlling  cost  [See  diagrams  labeled  Figure  1  and 
Figure  4  below].  Had  the  DIVAD  team  used  such  an  approach,  the  truth  of  their  values 
and  perfonnance  would  have  come  to  light  much  sooner.  Rather  than  merely  comparing 
spending  over  time  to  the  budget  over  time,  the  DIVAD  team  should  have  looked  at  the 
amount  of  work  accomplished  and  the  capabilities  established  (the  “value”)  over  that 
time  period.  Their  failure  to  do  so  indicates,  once  again,  that  rather  than  value 
Inexpensive  solutions,  they  simply  valued  Inexpensive  programs.  They  therefore  settled 
for  the  illusion  of  Inexpensive,  with  disastrous  results. 
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(Figures  are  from  Wikipedia  entry  on  Earned  Value  Management,  accessed  31 

Oct  08:  http://en.wikipedia.org/wiki/Earned_value_management) 

Much  is  made  of  the  Army’s  decision  to  simplify  the  procurement  process, 
freeing  Ford  from  the  complexities  of  government  regulation  and  requirements.  However, 
the  Army’s  “hands  off’  policy  was  unfortunately  accompanied  by  an  “eyes  shut  and  ears 
plugged”  approach,  creating  the  appearance  of  simplifying  the  situation  in  the  short  term, 
while  actually  making  things  more  complicated  down  the  road.  There  is  a  difference 
between  not  interfering  with  the  contractor’s  decisions  and  not  really  being  aware  of 
them,  a  distinction  the  DIVAD  project  leaders  did  not  seem  to  grasp.  Simply  handing  a 
contract  to  a  company  and  asking  them  to  report  back  when  the  job  is  complete  is  not 
simple  -  it  is  simplistic,  which  is  quite  different  from  the  Simple  value. 

The  decision  to  incorporate  mature,  proven  technology  has  the  appearance  of 
simplicity,  but  not  the  substance.  This  approach  is  only  effective  if  the  technology 
selected  for  integration  into  the  system  actually  does  what  it  needs  to  do.  Thus,  using  the 
F-16  radar  creates  the  appearance  of  using  “mature,  proven  technology,”  except  for  the 
fact  that  the  F-16  radar  is  only  good  at  detecting  moving  targets,  while  the  DIVAD 
needed  to  acquire  stationary  ones.  Similarly,  building  the  DIVAD  onto  an  existing  tank 
chassis  would  only  make  sense  if  that  chassis  was  powerful  enough  to  haul  the  DIVAD ’s 
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full  weight.  In  fact,  “the  M-48’s  750-horsepower  engine  was  designed  to  power  a  50-ton 
tank,  not  the  60-ton  Sergeant  York  vehicle.”  (Adam,  1987:32)  Since  the  DIVAD’s  weight 
exceeded  the  engine’s  designed  capacity  by  20%,  one  might  argue  that  the  M-48,  while 
mature,  for  this  purpose  was  neither  proven  nor  appropriate.  Subsequent  testing  showed 
that  it  was  simply  not  up  to  the  task. 

Interestingly,  while  much  of  the  DIVAD’s  hardware  was  reuse,  the  software  was 
primarily  a  new  development.  Not  surprisingly,  it  was  plagued  with  errors.  Recall,  this 
development  took  place  in  the  early  1980’s,  when  computer  programming  was  a  much 
less  mature  discipline  than  it  is  today.  So,  the  DIVAD  team’s  decision  to  use  mature 
(albeit  inadequate)  hardware  is  overwhelmed  by  the  use  of  immature,  buggy  software. 

The  Tiny  value  is  the  least  ambiguous  of  the  four.  Not  much  is  said  about  the  size 
of  the  team,  but  the  vehicle  was  20%  larger  than  the  tank  from  which  it  got  it  chassis  and 
the  large  40mm  munitions  were  sufficiently  large  to  significantly  reduce  the  rate  of  fire, 
limiting  the  gun’s  effectiveness. 

The  emphasis  on  appearances  extends  to  the  operational  testing.  The  literature 
indicates  the  system  managed  to  shoot  down  9  out  of  1 1  helicopters,  which  sounds  like  an 
apparently  impressive  82%  kill  rate.  However,  since  “many  of  the  aircraft  had  to  make 
multiple  passes,”  (Adam,  1987:34),  the  kill  rate  was  actually  much  lower.  We  can 
conservatively  assume,  for  the  sake  of  argument,  that  fewer  than  half  the  targets  made 
one  extra  pass  each,  for  a  total  of  5  additional  passes.  The  DIVAD  then  would  have  a  kill 
rate  of  9  out  of  16,  for  a  much  less  impressive  56%  kill  rate.  The  actual  rate  is  unknown, 
but  probably  less  than  56%. 
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In  order  to  be  effective  and  influential,  guiding  values  must  be  deeply  held.  The 
FIST  values,  properly  understood  and  implemented,  go  to  the  core  of  each  project  leader 
and  team  member.  Settling  for  the  mere  appearance  of  the  FIST  values,  cutting  corners 
and  creating  the  illusions  of  speed  and  thrift  -  these  are  the  antithesis  of  the  FIST  value 
set.  The  DIVAD  project  shows  no  evidence  of  the  depth  typical  in  projects  with  a  high 
FIST  score,  despite  its  surface-level  similarities. 

The  program  was  cancelled,  and  DIVAD  is  often  pointed  to  as  an  “excellent 
example  of  many  of  the  problems  with  modem  weapon  acquisition  programs.”  (Ditton, 
1988:6).  DIVAD’s  failure  was  so  complete  and  widely  publicized  that  US  Senator  Jim 
Sasser  (D-TN)  asked  that  the  project  be  known  solely  as  DIVAD  rather  than  Sergeant 
York,  out  of  respect  of  his  state’s  prominent  World  War  I  hero. 
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